Collecting Microsoft Teams Logs

Microsoft Teams has three different types of logs, which get stored on different locations.

Web Logs
Consists of most Teams client activity.

• Windows: to capture logs, open Teams and press CTRL+ALT+SHIFT+1 in the client, and the logs will be downloaded to %downloads%\MSTeams Diagnostics Log.txt
• MAC: press Command+Option+SHIFT+1 in client to download logs to Downloads\MSTeams Diagnostics Log.txt

Desktop Logs
This log has most information about framework and bootstrapping information, app bootstrap process, plugin initialization, update management and SSO/ADAL Sign in information.

• Windows: %appdata%\Microsoft\Teams\logs.txt
• MAC: ~/Library/Application Support/Microsoft/Teams/logs.txt

Media Stack Logs
This log has media connectivity related information.

• Windows: %appdata%\Microsoft\Teams\media-stack
• MAC: ~/Library/Application Support/Microsoft/Teams/media-stack

Microsoft Teams Mobile client (iOS):
To collect log on Teams mobile client, open Microsoft Teams app -> Settings -> Report an issue. A new email will open with log.txt attached.

Determine Client Used to Send Email

Just recently someone asked me if there was a way to determine which email client (Outlook, OWA or ActiveSync) was used to send a particular email. On top of that, this person was also interested is finding out how many emails are sent per day using each of these clients.

The good news is that the Message Tracking Logs register this information. Every email sent has a SourceContext property which contains, amongst other information, the ClientType used to send the email. The important thing is to check this property for SUBMIT events, i.e., when the Mailbox Transport Submission service successfully transmits the email to the Transport service.

For SUBMIT events, the SourceContext property contains the following details:

• MDB: the mailbox database GUID;
• Mailbox: the mailbox GUID;
• Event: the event sequence number;
• MessageClass: the type of message. For example, IPM.Note;
• CreationTime: date and time of the message submission;
• ClientType: for example OWA or ActiveSync.

Please note that this only applies to emails sent by internal users. There is no SUBMIT event when an external sender sends an email to an internal user, meaning there is no ClientType property for these emails. In these cases, the only information we have regarding the sender is what the email headers contain, which does not include email client information.

To check what email client was used to send a particular email, we can run something like the following cmdlet and look at the SourceContext field:

Get-TransportService | Get-MessageTrackingLog -ResultSize Unlimited -Start 05/11/2016 -EventID SUBMIT -Sender nuno@nunomota.pt -MessageSubject Test | Select SourceContext

This field will contain information like the following:

MDB:34f3dc86-91bb-4ee7-a6a5-3d3ddc536050, Mailbox:a1de664f-9826-43a3-b9c8-3db019c86d8b, Event:29647741, MessageClass:IPM.Note, CreationTime:2016-05-11T07:17:14.922Z, ClientType:MOMT

In this case, MOMT stands for MAPI on the Middle Tier, basically clients that connect using Outlook or any other application that connects using RPC/HTTP or MAPI/HTTP.

To count the number of emails sent using OWA today, we can run something like this:

(Get-TransportService | Get-MessageTrackingLog -ResultSize Unlimited -Start 05/11/2016 -EventID SUBMIT | Where {$_.SourceContext -match "OWA"}).Count

Easy as that! :)

Monitor Mailbox Database Transaction Logs

Excessive database and/or transaction log growth is, unfortunately, not an uncommon problem in Exchange deployments. On top of being hard to troubleshoot, if it is found too late, it can cause serious issues for users and even the business. As such, it is crucial to have an adequate monitoring solution. However, this is not the case in every single organization, so I decided to write a basic script to keep an eye on the number of logs being generated across databases.

 

You can, for example, run the script every hour and if any database currently has more logs than a specified threshold, it sends an alert by email with the number of transaction logs for all databases, highlighting the one(s) that triggered the alert, and the current free space for the database (you might need to update it depending on whether you have the .edb file and logs on the same or different locations).

 

The script simply counts all files in the LogFolderPath location (variable for each database) as this makes it quicker than only looking for *.log files, and still be accurate for what we want to achieve.

You can download the complete final script from the TechNet Script Gallery.

Search Admin Audit Log Old Properties

When searching the Admin Audit Log using the Search-AdminAuditLog cmdlet, you might find some useful information missing:

Search-AdminAuditLog -Cmdlets Set-TransportService

ObjectModified     : EXAIO
CmdletName         : Set-TransportService
CmdletParameters   : {MessageTrackingLogMaxAge, MessageTrackingLogMaxDirectorySize, Identity}
ModifiedProperties : { }
Caller             : nunomota.pt/Users/Admin
Succeeded          : True
Error              : None
RunDate            : 9/10/2015 8:47:48 AM
OriginatingServer  : EXAIO (15.00.1104.000)

From the above, we can see what was changed (the maximum age for the logs and the maximum allowed size for the directory) but not what these settings were changed to...

To see this, we have to dig deeper into CmdletParameters:

(Search-AdminAuditLog -Cmdlets Set-TransportService).CmdletParameters

Name                                 Value
----                                 -----
MessageTrackingLogMaxAge             45.00:00:00
MessageTrackingLogMaxDirectorySize   10 GB (10,737,418,240 bytes)
Identity                             EXAIO

Now we know exactly what the user Admin changed! But what about if we want to know what these settings were before this change?

By default, the administrator audit log records only the cmdlet name, cmdlet parameters (and values specified), the object that was modified, who ran the cmdlet, when the cmdlet was run, and on what server the cmdlet was run. The administrator audit log does not log what properties were modified on the object. If we want the audit log to also include the properties of the object that were modified, we need to enable verbose logging by setting the LogLevel parameter to Verbose:

Set-AdminAuditLogConfig –LogLevel Verbose

When we enable verbose logging, in addition to the information logged by default, the properties modified on an object, including their old and new values, are included in the audit log:

(Search-AdminAuditLog -Cmdlets Set-TransportService).ModifiedProperties

Name                                 NewValue      OldValue
----                                 --------      --------
MessageTrackingLogMaxAge             45.00:00:00   31.00:00:00
MessageTrackingLogMaxDirectorySize   10 GB         5 GB

Now we know exactly what got changed and what the old configuration was!

Managed Availability Failed by Design Error

If you navigate to the Exchange’s ProbeResult Crimson Channel under ActiveMonitoring, you will likely find the following error event logged:

EventID: 2
Channel: Microsoft-Exchange-ActiveMonitoring/ProbeResult
ResultId: 479968430
ServiceName: Monitoring
ResultName: HealthManagerHeartbeatProbe
Error:  Failed by design.
Exception: System.Exception: Failed by design. at Microsoft.Office.Datacenter.ActiveMonitoring.TestActiveMonitoringProbe.DoWork(CancellationToken cancellationToken) at Microsoft.Office.Datacenter.WorkerTaskFramework.WorkItem.Execute(CancellationToken joinedToken) at Microsoft.Office.Datacenter.WorkerTaskFramework.WorkItem. c__DisplayClass2.startexecuting b__0() at System.Threading.Tasks.Task.Execute()

So why is something failing by design?! Basically this probe is testing the Managed Availability framework, including that it can run responders. More specifically, a probe that will test that monitors can become unhealthy and execute null responders.

Bottom line, nothing to worry about :)

Exchange 2013 Loose Truncation

Loose Truncation is a new feature that was introduced in Exchange 2013 Service Pack 1. Its purpose is to prevent possible disk space issues that can occur in environments with DAGs when one or more copies of a database is offline for an extended period of time. When enabled, loose truncation changes the “normal” truncation behavior. Each database copy tracks its own free disk space and starts to truncate transaction log files independently if the available disk space falls behind a set threshold configurable by the administrator.

To continue reading, please check the full Exchange 2013 Loose Truncation article at MSExchange.org.

Exchange Cmdlet History in Exchange 2010/2013

Having a list of all Exchange cmdlets we ran is always useful. Besides the usual methods of getting these from the PowerShell window history (by pressing the F7 key) or from the administrator audit logs, for example, the Event Log also has a list of these cmdlets!

Both in Exchange 2010 and 2013, most cmdlets are recorded under the Applications and Services Logs -> Microsoft -> MsExchange Management event log.

In here we have entries under the MSExchange CmdletLogs source and with an EventID of 1 (cmdlet ran successfully) and/or 6 (cmdlets that failed).

For example, by running the following cmdlet: Add-MailboxPermission Mota –User Nuno –AccessRights FullAccess –InheritanceType All

We get this entry in the event log: Cmdlet Add-MailboxPermission, parameters {Identity=Mota, User=Nuno, AccessRights={FullAccess}, InheritanceType=All}.

NOTE: cmdlets started with Get-* do not get logged.

Exchange Admin Center Cmdlet Logging

The Exchange 2010 Management Center included PowerShell cmdlet logging functionality, something that was removed in Exchange 2013. The good news is that this functionality has returned as part of the Exchange Admin Center in Exchange 2013 SP1 and soon to Exchange Online as well.

 

When you sign in with Admin credentials you will find the entry point for the PowerShell log view in the drop down list besides the help question mark under “Show Command Logging” (this feature is not available for non-admins):

 

The cmdlet list will display up to 500 entries and includes features such as search, export, start/stop logging, and more.

Message Tracking Report

New to Exchange 2013, the Get-MessageTrackingReport cmdlet is used to return data for a specific message tracking report.

 

This cmdlet, used by the delivery reports feature, requires you to specify the ID for the message tracking report you want to view. Therefore, first you need to use the Search-MessageTrackingReport cmdlet to find the message tracking report ID for a specific message. You then pass the report ID to the Get-MessageTrackingReport cmdlet.

 

Note that you need to be assigned permissions before you can run this cmdlet. You will not have access to some of its parameters if the account used is not a member of one of the following groups: Organization Management, Records Management or Recipient Management.

 

 

This first example gets the message tracking report for messages sent from one user to another and returns the summary of the message tracking report for a message that Alice Jones sent to John Richardson:

$Temp = Search-MessageTrackingReport “Alice Jones” -Recipients “johnr@letsexchange.com”
Get-MessageTrackingReport $Temp.MessageTrackingReportID -ReportTemplate Summary

• The ReportTemplate parameter specifies a predefined format for the output. You can either return a summary for all recipients or a detailed tracking report for one recipient using one of the following values: RecipientPath or Summary.

 

 

The second example gets the message tracking report for the following scenario: a user named Nuno Mota was expecting an e-mail message from joe@domain.com that never arrived. He contacted the Help Desk, which generated a message tracking report on behalf of Nuno returning detailed troubleshooting information for the specific recipient path:

Search-MessageTrackingReport “Nuno Mota” -Sender “joe@letsexchange.com” -ByPassDelegateChecking -DoNotResolve | ForEach {Get-MessageTrackingReport $_.MessageTrackingReportID -DetailLevel Verbose -BypassDelegateChecking -DoNotResolve -RecipientPathFilter "nunom@domain.com" -ReportTemplate RecipientPath}

• The BypassDelegateChecking switch allows Help Desk staff and administrators to retrieve message tracking reports for any user. By default, each user can only see the message tracking reports for messages sent or received by the user. When using this switch, Exchange allows administrators to view message tracking reports for messages exchanged among other users.
• The DoNotResolve switch prevents the resolution of e-mail addresses to display names. This improves performance, but the end result may not be as easy to interpret because it is missing the display names.
• The DetailLevel parameter specifies the amount of detail to be displayed for the message tracking report. You can use one of the following values: Basic or Verbose. If you specify Basic, simple delivery report information is displayed, which is more appropriate for information workers. If you specify Verbose, full report information is displayed, including server names and physical topology information.
• The RecipientPathFilter parameter specifies the recipient for which the command returns the detailed tracking report. This parameter is used when using the RecipientPath report template.

Exchange 2013 E-mail Traffic From inboundproxy@inboundproxy.com and HealthMailbox

In Exchange 2013, native, built-in monitoring and recovery actions are included in a feature called Managed Availability, which is made up of two processes: the Microsoft Exchange Health Manager Service (MSExchangeHMHost.exe) and the Microsoft Exchange Health Manager Worker process (MSExchangeHMWorker.exe), and the following components:

• The Probe Engine takes measurements on the server;
• The Monitoring Probe Engine stores the business logic about what constitutes a healthy state. It functions like a pattern recognition engine, looking for patterns and measurements that differ from a healthy state, and then evaluating whether a component or feature is unhealthy;
• The Responder Engine, when alerted about an unhealthy component, first tries to recover that component. The first attempt may be to restart the application pool, the second attempt may be to restart the corresponding service and the third attempt may be to restart the server. The final attempt may be to put the server offline, so that it no longer accepts traffic.

Exchange 2013 automatically creates several HealthMailbox"guid" objects in Active Directory which are used by Managed Availability to send e-mails through Exchange to verify mail flow every few minutes. These e-mails are used to do health checks for resources from FrontEnd Transport service to Transport Service and health checks on mailbox database resources. The same Microsoft Exchange Health Manager Service is responsible for both these health checks.

Two health mailboxes are created for each mailbox database: one for mailboxes and one for Public Folders (if these are deployed). You can view these hidden mailboxes using Active Directory Users and Computers. You need to enable Advanced Features and then navigate to Microsoft Exchange System Objects and then Monitoring Mailboxes:

Or you can use the Exchange Management Shell and run the following cmdlet:

Get-Mailbox -Monitoring

 

 

This is why in an Exchange 2013 environment you will see lots of traffic from the e-mail addresses inboundproxy@inboundproxy.com, MailDeliveryProbe@MailDeliveryProbe.com and HealthMailbox(...)@domain.com with the subjects of “MBTSubmission/StoreDriverSubmission/(...)-MapiSubmitLAMProve”, “Inbound proxy probe” or “Client submission probe”.

This is by design!

RPC Client Access Throttling Logging

By default, you have to use PerfMon counters to see how often throttling is occurring for RPC connections.

However, there is a way to have this information "properly" logged. To achieve this, modify the Microsoft.Exchange.RpcClientAccess.Service.exe.config file located in \Program File\Microsoft\Exchange Server\V14\Bin and add Throttling to the LoggingTag string so it looks like this:

<add key=”LoggingTag” value=”ConnectDisconnect, Logon, Failures, ApplicationData, Warnings, Throttling”/>

Now restart the RPC Client Access service and you will see this information in logs created in the “\Program Files\Microsoft\Exchange Server\V14\Logging\RPC Client Access” folder. When throttling happens, you will usually find the keyword “exceeded”.

This way, you can easily see when RPC Throttling kicks in, what it is doing and why!

Log Parser Studio

I have been using Log Parser for some years now and sometimes it is a life saver! It is easy to use and makes parsing logs so simple. However, in order to create some nice graphs you have to either write a more complex query or export all the data to Excel and then do it from there.

Until now! “With Log Parser Studio (LPS) we can house all of our queries in a central location. We can edit and create new queries in the ‘Query Editor’ and save them for later. We can search for queries using free text search as well as export and import both libraries and queries in different formats allowing for easy collaboration as well as storing multiple types of separate libraries for different protocols.”

For more information:

• Introducing: Log Parser Studio
• Download

Source: TheExchange Team Blog

Message Tracking Log Path

Here is a problem I faced today: when trying to move all the tracking logs for 3 CAS boxes from the C to the D drive I used the command:

Get-TransportServer Set-TransportServer -MessageTrackingLogPath D:\LogFiles\MessageTracking

At the time, I didn’t know that if the folder doesn’t exist Exchange will automatically create it. So, I manually created them, copied all the logs to the new location (because existing logs will remain in their previous location and new ones will be created in the new location) and immediately ran the command. After a couple of minutes I checked the Date Modified for the last log on the new folder and that showed me that Exchange was using the new folder. Ok, it’s done! :)
Not exactly...

When I went to the Application Log on the Event Viewer, I saw a few of these errors there:


Event Type: Error
Event Source: MSExchange Common
Event Category: Logging
Event ID: 6004
Date: 16/04/2009
Time: 09:20:32
User: N/A
Computer: DS-EXCAS3
Description: MessageTrackingLogs: Failed to write logs because of the error: Access to the path 'MSGTRK20090317-1.LOG' is denied..


What?! That’s from one month ago!
Ok, let’s look at my current configuration:

MessageTrackingLogEnabled : True
MessageTrackingLogMaxAge : 30.00:00:00
MessageTrackingLogMaxDirectorySize : 2GB
MessageTrackingLogMaxFileSize : 20MB
MessageTrackingLogPath : D:\LogFiles\MessageTracking
MessageTrackingLogSubjectLoggingEnabled : True


Here’s the problem: as you can see, I have it so that Exchange only keeps 30 days of tracking logs. Since I manually created the new folders, I didn’t set up the correct permissions on them. So, Exchange couldn’t delete that log from one month ago! It was creating new ones and logging everything, but just couldn’t delete that one...

If you create the target folder for the logs automatically when running the command, Exchange will assign the correct permissions to the folder. But If you manually create it, make sure you give the following permissions to it:

NETWORK SERVICE: Modify, Read & Execute, List Folder Contents, Read and Write (also give it the Delete Subfolders and Files permission!)


Just out of curiosity, this is the command I used to configure the Message Tracking Logs for my CAS boxes:

Get-TransportServer Set-TransportServer -MessageTrackingLogEnabled $True -MessageTrackingLogPath “D:\LogFiles\MessageTracking” -MessageTrackingLogMaxAge “30.00:00:00” -MessageTrackingLogMaxDirectorySize 2GB -MessageTrackingLogMaxFileSize 20MB -MessageTrackingLogSubjectLoggingEnabled $True


Hope this helps!