ePolicy Orchestrator 5.1 or 5.3 server tasks do not expire after reaching abort time

ePolicy Orchestrator 5.1 or 5.3 server tasks do not expire after reaching abort time

Environment
McAfee ePolicy Orchestrator (ePO) 5.3.0, 5.1.1, or 5.1.0

Problem

If ePO is unable to connect to a client system, certain server tasks (including Agent Wakeup Call, Agent Deployment, and AD sync) may remain in an In Progress state indefinitely.

This issue occurs when an abort after time is configured in the task. When the abort after time is exceeded, ePO fails to expire the task. This issue can occur with ePO 5.3.0, 5.1.1, or 5.1.0. 

IMPORTANT: There is another very similar issue with these same symptoms in ePO 5.1.0, referenced in KB79875. If you use ePO 5.1.0 and experience these same symptoms, ensure that you have applied the hotfix referenced in KB79875 before following the steps in this article. Alternatively, upgrade to ePO 5.1.1 first, and if the issue persists, apply the steps in this article.

Problem

The ePODataChannelData table may become very large in size. This is because old entries are removed from that table by the maintenance task, which is no longer running. Upon correcting the issue, if a very large number of entries do exist in this table, it may take several hours for the maintenance task to purge the orphaned entries. During this time the ePO Data Channel may be very slow or non-functional. 

System Change

The time was changed back to an earlier time on the ePO server. For example, you may have observed this issue soon after the Daylight Savings Time (DST) change that occurred in the United States on November 1, 2015.

The Daylight Savings Time change occurs on different dates for different regions, so it is possible for this issue to start occurring on a different date for ePO servers that reside in a different country. For example, in Mexico City, Daylight Savings Time occurred on October 25, 2015.

Cause

This issue is caused by a problem with the DB-Cleanup internal server task. The DB-Cleanup task expires other server tasks that have not completed by the configured abort time. This task should run every minute. However, after a time change, this task could stop executing.

Solution

This issue is resolved in ePO 5.1.2, which is available from the Product Downloads site at: http://mcafee.com/us/downloads/downloads.aspx.

NOTE: You need a valid Grant Number for access. KB56057 provides additional information about the Product Downloads site, and alternate locations for some products.

Updates are cumulative; Technical Support recommends that you install the latest one.


This issue is resolved in ePO 5.3.1, which is available from the Product Downloads site at: http://mcafee.com/us/downloads/downloads.aspx.

NOTE: You need a valid Grant Number for access. KB56057 provides additional information about the Product Downloads site, and alternate locations for some products.

Updates are cumulative; Technical Support recommends that you install the latest one.


After you apply the scripts listed in Solution 2, the issue should not occur again unless you change the time on the ePO server back to an earlier time.

The issue does not occur if you change the time forward. Therefore, the Spring DST time change that will occur in March 2016 should not cause the issue.

This issue should only occur during a time change from Daylight Time to Standard time, and only if you have not upgraded the ePO server to 5.1.2 (or later) or 5.3.1 (or later) by then.

Solution

NOTE: It is possible this same issue may occur again if you change to an earlier time on the ePO server. If this happens, you can reapply the following steps.
  1. Back up the ePO database. See KB52126 for details about how to back up and restore the ePO database using SQL Server Management Studio.
  2. Stop the ePO services:
    1. Click Start, Run, type services.msc, and click OK.
    2. Right-click each of the following services and click Stop:
       
      • McAfee ePolicy Orchestrator 5.x Application Server
      • McAfee ePolicy Orchestrator 5.x Event Parser
      • McAfee ePolicy Orchestrator 5.x Server 
         
  3. Run the following SQL statements against the ePO database using SQL Server Management Studio:
    1. Open SQL Management Studio.
    2. Expand Databases, right-click on the ePO database, and click New Query.
    3. Enter the following commands in the blank area: 

      update OrionTaskQueueMT set RunTime=GETUTCDATE() where taskdescription like '%dbclean%' 
      update OrionTaskQueueMT set RunTime=GETUTCDATE() where taskdescription like '%task.queue%'  
       
    4. Click Execute to run the query. 
       
  4. Restart the ePO services:
    1. Click Start, Run, type services.msc, and click OK.
    2. Right-click each of the following services and click Start: 

      McAfee ePolicy Orchestrator 5.x Application Server 
      McAfee ePolicy Orchestrator 5.x Event Parser 
      McAfee ePolicy Orchestrator 5.x Server       
       
  5. Run the following query in SQL Server Management Studio to verify that the solution worked:
    1. Open SQL Management Studio.
    2. Expand Databases, right-click on the ePO database, and click New Query.
    3. Enter the following command in the blank area: 

      Select * from OrionTaskQueueMT where taskdescription like '%dbclean%' or taskdescription like '%task.queue%' 
       
    4. Click Execute to run the query. 

      When the query results are returned, look for the column named EnqueuedTime and the column named RunTime. If you refresh the above query every minute, you should notice that both the EnqueuedTime and RunTime are changing every minute. If so, the issue is resolved and you should no longer see server tasks stuck in an In Progress state.