source:
http://www.symantec.com/connect/articles/troubleshooting-microsoft-message-queuing-enterprise-vault

Troubleshooting Microsoft Message Queuing in Enterprise Vault.

The Don

August 6th, 2009

Filed under: Enterprise Vault, Tip/How to, Backup and Archiving

Troubleshooting Microsoft Message Queues with Enterprise Vault.

Explanation of each queue

Enterprise Vault Exchange Mailbox task for server queue A1

Pending Items that are being updated. Sometimes referred to as post processing, where items are converted from pending to fully archived in user’s mailboxes.

Enterprise Vault Exchange Mailbox task for server queue A2

Individual items to process. Used for manual archive requests and whenever Enterprise

Vault cannot directly communicate with the Storage Archive queue of the Storage service.

Enterprise Vault Exchange Mailbox task for server queue A3

Used whenever a Run Now operation has been executed on the Archiving Task. Lists mailboxes to process.

Enterprise Vault Exchange Mailbox task for server queue A4

Individual items to process. Only used for retries when Enterprise Vault cannot directly

communicate with the Storage Archive queue of the Storage service.

Enterprise Vault Exchange Mailbox task for server queue A5

Mailboxes to process. Used during background archiving. This queue is not processed outside the

scheduled archiving times, so you cannot use Run Now to clear a backlog on this queue.

Enterprise Vault Exchange Mailbox task for server queue A6

Requests to update folders with items that have been moved inside a mailbox. In older versions of Enterprise Vault than version 8, this queue processed synchronization.

Enterprise Vault Exchange Mailbox task for server queue A7

Synchronization Requests. This queue was not included pre-EV8.

Enterprise Vault Exchange Mailbox task for server queue R1

Items to retrieve

Enterprise Vault Exchange Mailbox task for server queue R2

Items ready for retrieval.

Enterprise Vault Spool Queue

Items restored by the Storage service that are now ready to be processed by the Exchange Mailbox task (for retrieval).

Enterprise Vault Exchange Mailbox task for server queue J1

Shortcuts to update and failed operations.

Enterprise Vault Exchange Mailbox task for server queue J2

Items to process.

Enterprise Vault Exchange Mailbox task for server queue J3

Instructs the Exchange Journaling task to examine the journal mailbox for new messages. Up to 250 new messages will be marked as archive pending and a message is placed on queue j2 for each such message.

Enterprise Vault Storage Archive for Exchange Server

Messages to be stored in Vault Store. These messages are divided up into chunks.

Enterprise Vault Storage Restore for Exchange Server

Messages to be restored to Mailbox. These messages are also divided up into chunks.

Things to check and ensure are configured correctly before troubleshooting.

You need to check the system is configured correctly in the first place before proceeding with any troubleshooting. Below is a check list that should assist.

1: Are MSMQ’s installed on a non-system drive as recommended? This can be checked by opening the properties of Message Queuing via Computer Management and reviewing the Storage TAB. The following technote would assist with moving these. http://seer.entsupport.symantec.com/docs/284889.htm

2: Are the MSMQ’s installed in workgroup mode as required? Check via Add/remove Programs and Components.

3: What is the maximum size configured, again contained in properties of Message Queuing via Computer Management and reviewing the General TAB.

4: Ensure you have installed at least Windows 2003SP2, as this contains a huge amount of fixes from Microsoft for message queues.

5: Ensure the MSMQ Storage Location is excluded by Anti-Virus, both scheduled and on-access scans. Failure to do this will cause problems with Enterprise Vault and MSMQ’s.

General Troubleshooting of MSMQ’s

Check the System and Application Event for any errors relating to MSMQ’s.

Identify what queue for example is not processing to try and give you an indication of the issue. For example if the Enterprise Vault Exchange Mailbox task for server queue A1 is not processing then the issue that will occur is that items will not be converted to shortcut in user’s mailboxes and will remain pending until cancelled or timed out. Allot of the time monitoring MSMQ’s will alert you to other issues. For example if items are not being post processed and never placed on the A! queue for an Exchange Server then the issue is most likely the items have not been backed up.

You should also note what aspects of MSMQ’s appear to not be working. Is the issue, only occurring on one Exchange Server’s MSMQ’s? Which queue has the issue? Are other queues working? Can you see items being processed via the queue?

Sometimes enabling the Journaling feature in MSMQ’s can assist with capturing and showing items being processed. To access this open the properties of the queue concerned and enable journaling. A sub-folder under this will show the items captured in journaling. I always find it useful to ensure I have set a limit when I enable this feature, as without this, it is easy to forget that this has been set untul your disk is full!

Also check the permissions on the actual storage location, where the core files are held. Are MQ files being written?

Sometimes it is quicker and easier to simply reinstall the MSMQ’s to resolve issues. The instructions for this are below.

1. Confirm that the Message Queues are empty and determine the MSQM storage location

a. Log in as the Vault Service Account

b. Launch Computer Management

c. Expand "Services and Applications"

d. Determine the location of the MSMQ files. Right-click, Properties of the icon "Message Queuing". Choose the Storage tab and make note of the storage locations. Default is "c:\windows\system32\msmq\".

e. Expand "Message Queuing"

f. Expand "Private Queues" - check to make sure if these are empty. If not, then make sure you understand the ramifications of clearing the queues before doing so.

2. Stop the Enterprise Vault services.

3. Stop the Message Queuing service.

4. Rename the old message queue directory to "msmq_old"

5. Uninstall the message queue from 'Add/Remove Programs - Windows components'

a. This should remove the old directory if it still exists delete the directory.

6. Reinstall the message queues (select 'Common', Active Directory will be selected by default, but is not needed - and causes a performance hit)

7. The Message queue service should automatically start, but if it is not started, start it up.

8. Confirm the newly created msmq directory exists.

9. Restart the Enterprise Vault services

a. The missing message queues will be recreated in "Private Queues" when the Tasks begin to process.

10. Delete the "msmq_old" directory created earlier to reclaim disk space.

a. No reboot should be necessary.

Message queues are recreated by the Enterprise Vault Services. Therefore, you could also, after ensuring the queues are empty, remove these by right clicking and deleting, then restart the Enterprise Vault services. This should recreate the necessary queues.

If you purge data in MSMQ’s (you should take care doing this) you may receive messages in the Enterprise Vault Event Log relating to negative acknowledgements which is normal and can be ignored. Refer to the following technote for further details. http://support.veritas.com/docs/276082

You should also be aware that when monitoring MSMQ’s through a cluster, you will need to connect via Computer Management to the virtual cluster name not the physical name of the nodes. You can right click on Computer Management when this is open in a MMC and select connect to.

There are a number of useful technotes relating to MSMQ’s on the Enterprise Vault Support site, and this should well be worth reviewing before logging a case with Symantec. Refer to http://support.veritas.com