Skip to content

EA MtApi5 get locked/frozen under mutli-symbol optimization mode during backtesting #261

Closed
Closed
EA MtApi5 get locked/frozen under mutli-symbol optimization mode during backtesting#261

Description

@joecklau
joecklau
opened on Aug 14, 2021

Situation:

When running backtesting on MtApi5 with multiple symbols optimization mode, once the last tick of a symbol within the date-range is proceeded, MtApi5 will stay frozen. No connection/command will be able to sent to MtApi5 at this status.

Expected behaviour:

When trying backtesting multiple symbols optimization mode on other default sample EA, once all ticks of a symbol are proceeded, the ticks of next symbols will be started proceeded until the last symbol is also proceeded.

How to reproduce the situation:

  • Ensure the market watch has several/multiple symbols selected.
  • Optimization = "All symbols selected in market watch"
  • Set Date to custom period, pick a few days for fewest data to reach "the last tick" status.
  • For input, any BacktestingLockTicks mode can get the same result, but I usually picked "LOCK EVERY TICK"
  • For Agents, I disabled all other cores but leaving only one core enabled for the testing as I use 1 port only. (I guess only 1 agent/core can be supported for backtesting on MtApi5 consider the port connection setting?)

Then start the backtesting, wait a for a moment until the last tick of 1st symbol is reached. Then it will stay in frozen status.

What's also tested:

I tried re-connecting to the MtApi5 by the dotnet dll via the port, but unlike normal situation which was be ok to reconnect, it will be null response when it's under the frozen status.

Activity

  1. joecklau commented on Aug 16, 2021

    @joecklau
    joecklau
    on Aug 16, 2021
    ContributorAuthor

    More testings:
    For first iteration on a symbol, Inside MQL, OnInit() are OnDeinit(const int reason) are invoked successfully.
    After the first iteration, the EA starts OnInit() but stays inside "wait for command (BacktestingReady) from remote side to be ready for work" waiting loop.

    At this moment, inside C# side, unlike first iteration, it's very difficult for MtApi5Client.BeginConnect(port) to connect to the MtApi5 EA. Even the MtApi5Client.ConnectionState sometimes changed to CONNECTED after some retrials, commands from MtApi5Client dll towards MtApi5 EA always get "Response from MetaTrader is null" at this stage.

    I doubt maybe some resource was not completely released/closed inside MT5Connector.dll -> deinitExpert(int expertHandle, string& err)
    So that the MtApi5Client cannot re-connect via the same port for next iteration?

  2. joecklau commented on Aug 16, 2021

    @joecklau
    joecklau
    on Aug 16, 2021
    ContributorAuthor

    Root cause is found, but I think my handling is not suitable to create pull request, better just put my findings and dirty handling here as reference to @vdemydiuk . It's about _stopTimer and the handling in _stopTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e) inside /MTApiService/Mt5Expert.cs

    In MT5 multiple symbol optimization backtesting mode, when one symbol's processing end and MT5 start another symbol's backtesting, it looks very likely violating the situation "Mt5Expert has received new tick during 2 sec in testing mode", thus invoking Deinit() unexpectedly to me.

    As it's just local machine backtesting, my dirty handling is only comment-out this lines, then it works:

                //Log.Warn("Mt5Expert has received new tick during 2 sec in testing mode. The possible cause: user has stopped the tester manually in MetaTrader 5.");
            //Deinit();

    Here's the log before my handling as reference:

    2021-08-17 00:20:32,688 [1] INFO  MTApiService.MtAdapter - AddExpert: begin. expert = ExpertHandle = 0, Quote = Instrument = MSFT.NAS; Bid = 246.41; Ask = 246.42; ExpertHandle = 0
2021-08-17 00:20:32,801 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://221.168.1.40:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,803 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://172.21.0.204:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,805 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://221.168.1.41:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,807 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://172.21.144.1:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,813 [1] INFO  MTApiService.MtAdapter - AddExpert: end
2021-08-17 00:20:52,411 [1] INFO  MTApiService.MtAdapter - RemoveExpert: begin. expertHandle = 0
2021-08-17 00:20:52,421 [1] INFO  MTApiService.MtAdapter - RemoveExpert: end
2021-08-17 00:20:52,498 [1] INFO  MTApiService.MtAdapter - AddExpert: begin. expert = ExpertHandle = 0, Quote = Instrument = TSLA.NAS; Bid = 670.75; Ask = 670.83; ExpertHandle = 0
2021-08-17 00:20:52,499 [1] INFO  MTApiService.MtAdapter - AddExpert: end
2021-08-17 00:20:54,418 [9] WARN  MTApiService.Mt5Expert - Mt5Expert has received new tick during 2 sec in testing mode. The possible cause: user has stopped the tester manually in MetaTrader 5.
2021-08-17 00:20:54,461 [1] ERROR MTApiService.MtAdapter - The given key was not present in the dictionary.

  3. KptKuck commented on Aug 20, 2021

    @KptKuck
    KptKuck
    on Aug 20, 2021

    Situation:

    When running backtesting on MtApi5 with multiple symbols optimization mode, once the last tick of a symbol within the date-range is proceeded, MtApi5 will stay frozen. No connection/command will be able to sent to MtApi5 at this status.

    Expected behaviour:

    When trying backtesting multiple symbols optimization mode on other default sample EA, once all ticks of a symbol are proceeded, the ticks of next symbols will be started proceeded until the last symbol is also proceeded.

    How to reproduce the situation:

    * Ensure the market watch has several/multiple symbols selected.

* Optimization = "All symbols selected in market watch"

* Set Date to custom period, pick a few days for fewest data to reach "the last tick" status.

* For input, any BacktestingLockTicks mode can get the same result, but I usually picked "LOCK EVERY TICK"

* For Agents, I disabled all other cores but leaving only one core enabled for the testing as I use 1 port only. (I guess only 1 agent/core can be supported for backtesting on MtApi5 consider the port connection setting?)

    Then start the backtesting, wait a for a moment until the last tick of 1st symbol is reached. Then it will stay in frozen status.

    What's also tested:

    I tried re-connecting to the MtApi5 by the dotnet dll via the port, but unlike normal situation which was be ok to reconnect, it will be null response when it's under the frozen status.

    This API does not work with the optimization.
    When a new run starts the connection is disconnected.
    This can not work like this.

  4. joecklau commented on Aug 25, 2021

    @joecklau
    joecklau
    on Aug 25, 2021
    ContributorAuthor

    Situation:

    When running backtesting on MtApi5 with multiple symbols optimization mode, once the last tick of a symbol within the date-range is proceeded, MtApi5 will stay frozen. No connection/command will be able to sent to MtApi5 at this status.

    Expected behaviour:

    When trying backtesting multiple symbols optimization mode on other default sample EA, once all ticks of a symbol are proceeded, the ticks of next symbols will be started proceeded until the last symbol is also proceeded.

    How to reproduce the situation:

    * Ensure the market watch has several/multiple symbols selected.

* Optimization = "All symbols selected in market watch"

* Set Date to custom period, pick a few days for fewest data to reach "the last tick" status.

* For input, any BacktestingLockTicks mode can get the same result, but I usually picked "LOCK EVERY TICK"

* For Agents, I disabled all other cores but leaving only one core enabled for the testing as I use 1 port only. (I guess only 1 agent/core can be supported for backtesting on MtApi5 consider the port connection setting?)

    Then start the backtesting, wait a for a moment until the last tick of 1st symbol is reached. Then it will stay in frozen status.

    What's also tested:

    I tried re-connecting to the MtApi5 by the dotnet dll via the port, but unlike normal situation which was be ok to reconnect, it will be null response when it's under the frozen status.

    This API does not work with the optimization.
    When a new run starts the connection is disconnected.
    This can not work like this.

    MetaTrader Terminal has auto-update mechanism, which will auto-close-and-restart the terminals periodically. So disconnection handling is as-expected.
    My handling practice is re-connection. So the key for my usage is to ensure the disconnected EA/dll don't hold the port/resource unexpectedly.

    So far my dirty modification somehow allows me to do multi-symbol optimization, with random pauses every a number of symbols, which requires a few stop-and-start clicks to resume the progress...
    I'm sure there should be better modification on /MTApiService/Mt5Expert.cs rather than my dirty one above.

  5. KptKuck commented on Nov 21, 2021

    @KptKuck
    KptKuck
    on Nov 21, 2021

    Maybe this will help you.

    Do you know that you can start the strategy tester individually via the CLI ?
    In my project a MatLab program controls the strategy tester. there I am variable in the symbol,timeframe or start date and end date. Up to 4 instances run in parallel.
    Look in the documentation under:

    Metatrader advanced start.

    https://www.metatrader5.com/en/terminal/help/start_advanced/start

  6. joecklau commented on Nov 21, 2021

    @joecklau
    joecklau
    on Nov 21, 2021
    ContributorAuthor

    Maybe this will help you.

    Do you know that you can start the strategy tester individually via the CLI ? In my project a MatLab program controls the strategy tester. there I am variable in the symbol,timeframe or start date and end date. Up to 4 instances run in parallel. Look in the documentation under:

    Metatrader advanced start.

    https://www.metatrader5.com/en/terminal/help/start_advanced/start

    Interesting! Combining with some CLI and dynamic .ini file generation coding work, it seems possible to do many things instead of relying of optimisation option in UI!
    Not limited to MtApi, but any EA in MT5.
    Thanks for such an inspiring input!

  7. tudstudent commented on Mar 21, 2026

    @tudstudent
    tudstudent
    on Mar 21, 2026

    Fixed in PR #297 — Execute_UnlockTicks() in MT5 was missing _is_ticks_locked = false, causing the EA to spin forever in the OnTimer loop during multi-symbol backtesting. Also added EventSetMillisecondTimer(100) for live trading so commands process without waiting for ticks.

  8. vdemydiuk
    added a commit that references this issue on Apr 6, 2026

    Issue #261: MQL5 - reset flag _is_ticks_locked in function Execute_Un…

    a8bb1a3

  9. vdemydiuk commented on Apr 7, 2026

    @vdemydiuk
    vdemydiuk
    on Apr 7, 2026
    Owner

    My bad. I missed code line _is_ticks_locked = false in function Execute_UnlockTicks after refactoring.
    Now it is fixed.

  10. vdemydiuk
    closed this as completedon Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions