New Drive Shipping Technology in Microsoft Exchange Server 2016

4/2: The post below was posted as an April fools joke :)

Repost from:
https://www.facebook.com/groups/MSEX2013/permalink/1031304146898702/
by Boris Lokhvitsky

In the modern messaging world, there is a clear and apparent trend for increasing volume of messaging data communicated, accumulated and stored in Microsoft Exchange mailboxes. Growing mailbox capacity expedites adoption for large 6TB and 8TB hard drives used in the storage solutions for Microsoft Exchange. This also creates significant challenges for database replication, which is a cornerstone technology providing data redundancy and high availability to Exchange mailboxes. Despite the high bandwidth networks becoming more and more common for most customers, even 10 Gbps networks have challenges processing bandwidth intensive Exchange database replication and indexing.

It is an easy though not obvious math exercise to realize that the process of offline data shipping using hard drives as a carrier is capable to achieve much higher data throughput than any currently available network technologies can provide. For example, shipping just 1,000 8TB hard drives across the United States using UPS service takes only 5 business days (and could be further expedited using rush delivery option), which translates into the bandwidth of ~150 Gbps – far exceeding the capability of the most advanced WAN links available today!

Inspired by this consideration, Microsoft Exchange team started working to implement drive shipping as the new data replication technology. As you know, we already work on implementing drive shipping in Office 365 as an efficient mechanism to import PST files, which is provided as part of Azure Import Export Service:
http://www.msexchange.org/…/drive-shipping-and-network-base…

Today, we are happy to announce that in the upcoming Microsoft Exchange Server 2016 we will completely switch to the drive shipping technology. It will replace transaction log shipping currently used in Exchange DAGs for database replication. New replication process will be still controlled by Microsoft Exchange Replication service, which is completely reworked and now integrated with U.S. Postal Service (via postal.dll component) and with leading courier delivery services such as FedEx and UPS.

Only 3.5 inch SATA II/III hard drives are currently supported for use with the drive shipping service. Hard drives larger than 8TB are not supported. For import jobs, only the first data volume on the drive will be processed. The data volume must be formatted with NTFS.

Service agreement with existing Office 365 customers will include necessary supply of the drives required for drive shipping. For the on premises customers, Enterprise license for Exchange 2016 will include one-year service agreement with leading U.S. courier delivery services – FedEx and UPS.

Our product roadmap does not stop there. Microsoft is currently working to implement the new drive shipping transport protocols based on drone delivery. If successful, the drone delivery architecture might also replace the outdated and inefficient SMTP protocol currently still being used for e-mail transmission.

We hope that all customers realize the benefits and advantages of the new drive shipping technology and start enjoying new functionality coming in Exchange Server 2016!

by Boris Lokhvitsky
April 1, 2015

Posted in Exchange | Tagged | 2 Comments

KB3002657 v2 release for Windows 2003 – Fixes authentication issues on 2003 DCs

For details on this issue see my blog post here: Warning: KB3002657 can break authentication, with Exchange and other apps & devices

Microsoft added this note to the MS15-027 post

V2.0 (March 16, 2015): To address a connectivity issue with update 3002657 when installed on supported editions of Windows Server 2003, Microsoft released update 3002657-v2 for all supported editions of Windows Server 2003. Customers who have not already installed the 3002657 update should install update 3002657-v2 to be fully protected from this vulnerability. To avoid the possibility of future detection logic problems, Microsoft recommends that customers running Windows Server 2003 who have already successfully installed the 3002657 update also apply update 3002657-v2 even though they are already protected from this vulnerability. Customers running other Microsoft operating systems are not affected by this rerelease and do not need to take any action. See Microsoft Knowledge Base Article 3002657 for more information.

Download KB3002657 v2 here: x86 | x64 | Itanium

Posted in Technical, Windows | Tagged , , | 1 Comment

Exchange 2013 CU8, 2010 SP3 UR9, & 2007 SP3 UR16 Released

3/23 Update: AD Schema changes are not made if already running 2013 CU7
3/19 Update: Added info on Public Folder scalability improvements

Exchange 2013 CU8, 2010 SP3 CU9, and 2007 SP3 UR16 were released on 3/17/2015

2013 CU8 | Download | KB3030080 | EHLO Post
2010 SP3 UR9 | Download | KB3030085| EHLO Post
2007 SP3 UR16 | Download | KB3030086| EHLO Post
Some of the above KBs may not be available yet, but should be within 24 hours

Key changes in 2013 CU8

  1. Public Folders
    1. Scalability improvements
      1. Up to 1 million Public Folders on 2013, a 400% increase from CU7!
        • Still limited to 100 PF Mailboxes and 100,000 Folders per PF Mailbox
      2. Up to 500,000 legacy (2007 or 2010) Public Folders being migrated to 2013
    2. Support for accessing Calendar & Contacts folders in OWA
    3. Adding favorite folders in Outlook
    4. Improved throughput and migration experience to 2013
      • Throughput is increased via multiple moves being carried at once, one per target Public Folder mailbox
  2. ActiveSync redirection to Office 365, after mailboxes are moved from on-premises to O365
    • Both Exchange 2013 CU8 and 2010 SP3 UR9 include this support
  3. Updates to AD schema
    • Only if running 2013 CU6 or earlier, CU7 & CU8 have the same schema updates

Also see Tony Redmond’s post on this “boring” update: Exchange 2013 CU8 appears. Instant boredom ensues – but for the best possible reason which is a good thing :)

Exchange 2010 & 2103 ActiveSync with hybrid deployments improvements

With Exchange 2013 Cumulative Update 8 and Exchange 2010 SP3 RU9, improvements have been made to Exchange ActiveSync in a hybrid deployment with Office 365. Previously, when a mailbox was moved from an on-premises Exchange server to Office 365, the user’s mobile device would stop syncing with their mailbox. To resolve this issue, the user would need to reconfigure it, to point to m.outlook.com, or recreate the email account on their device.  With the release of Exchange 2013 CU8 and Exchange 2010 SP3 RU9, Exchange will now automatically redirect the mobile device, if it supports this, to Office 365 when the mailbox is moved. With only a few exceptions, the user no longer needs to manually set up their device for mail to keep working. In addition to Exchange 2013 CU8, automatic redirection is supported in the following scenarios:

  • Exchange 2007 on-premises organizations (not supported)
  • Exchange 2010 on-premises organizations where the following is true:
    • All Exchange 2010 Client Access servers are running at least Exchange 2010 SP3 UR9
    • A hybrid deployment was configured using the Exchange Hybrid Configuration wizard
  • Mix Exchange 2010 and Exchange 2013 on-premises organizations where the following is true:
    • A hybrid deployment was configured using the Exchange Hybrid Configuration wizard
    • All Client Access servers have been replaced with either Exchange 2013 CU8 Client Access servers (with at least one Exchange 2013 CU8 Mailbox server) or Exchange 2010 SP3 UR9 Client Access servers

If your organization meets the requirements listed above, mobile devices should automatically be redirected to Office 365 when a user’s mailbox is moved, without any additional configuration. For the best experience, make sure your users’ mobile devices are running the latest versions of their operating systems and e-mail clients. Some mobile devices, such as those running the Android operating system, might not correctly interpret the Exchange ActiveSync 451 redirection instructions sent by Exchange. For these devices, users will still need to manually reconfigure or recreate their email account on the device. If you have questions about whether a device supports Exchange ActiveSync 451 redirection, contact the device manufacturer.

Changes coming in CU9, that were initially on the list for CU8

OWA logoff behavior will change in CU9, see this EHLO article for more details: OWA Forms Based Auth Logoff Changes in Exchange 2013 Cumulative Update 9. This does affect clients using TMG, so read this article to be prepared for the changes coming in CU9.

Also coming in CU9, is the ability to have Sent Items from a shared mailbox to got the shared mailbox Sent Items folder vs. the sending user’s. See this EHLO article for more details: Want more control over Sent Items when using shared mailboxes?

Exchange 2007 SP3 UR16 only change is an update to time zones.

Known Issues

If major issues come up with these updates I will add a section on Known issues later. For Known Issues and updates in 2013 CU7 see: Exchange 2013 CU7, 2010 SP3 UR8, 2007 SP3 UR15 released & Known Issues

Posted in Exchange, Technical | Tagged , , , | 1 Comment

Warning: KB3002657 can break authentication, with Exchange and other apps & devices

Last Updated: 3/17/15: Microsoft added this note to the MS15-027 post

V2.0 (March 16, 2015): To address a connectivity issue with update 3002657 when installed on supported editions of Windows Server 2003, Microsoft released update 3002657-v2 for all supported editions of Windows Server 2003. Customers who have not already installed the 3002657 update should install update 3002657-v2 to be fully protected from this vulnerability. To avoid the possibility of future detection logic problems, Microsoft recommends that customers running Windows Server 2003 who have already successfully installed the 3002657 update also apply update 3002657-v2 even though they are already protected from this vulnerability. Customers running other Microsoft operating systems are not affected by this rerelease and do not need to take any action. See Microsoft Knowledge Base Article 3002657 for more information.

Download KB3002657 v2 here: x86 | x64 | Itanium

The v1 KB3002657 update, which addresses issues in MS15-027,  breaks authentication for some applications and devices that use NTLM for authentication. This includes SMB/SMB2/SMB3, used for file shares and NAS, and other clients. It can also break IIS integrated authentication, even if set to Basic per some reports. Your issues will vary depending on Domain Controller version, 2003 seems to be affected the most, and server OS.  This vulnerability is also known as CVE-2015-0005.

Windows 2003 DCs using NTLM authentication is affected by this update. There have been reports of other OS versions being affected, but those have not been confirmed.

MS15-027: Vulnerability in NETLOGON Could Allow Spoofing:

This security update resolves a vulnerability in Microsoft Windows. The vulnerability could allow spoofing if an attacker who is logged on to a domain-joined system runs a specially crafted application that could establish a connection with other domain-joined systems as the impersonated user or system. The attacker must be logged on to a domain-joined system and be able to observe network traffic.

In most cases, after installing this update on a Windows 2003 DC, users will be prompted for authentication over and over, without success. With Exchange it seems to break OWA, Outlook Anywhere, and even mail delivery (server to server authentication) in some environments.

Workaround: Install v2 of KB3002657 or switch to Kerberos authentication if possible. Download v2: x86 | x64 | Itanium

See this articles for more details

I was 1st made aware of this issue on the Exchange 2013 Information Sharing Group on Facebook, but also came up in mailing list I follow. This update, KB3002657, causes authentication issues with SharePoint, Exchange, SQL, and more. Mainly it also breaks AD authentication against Windows 2003 domain controllers.

In addition, this update may break authentication with other systems\applications, for example the EMC Isilon and Dell FS Series NAS can fails to authenticate. Microsoft has included the following note on the KB:

SMB/SMB2/SMB3 clients may experience logon failures to an EMC Isilon cluster when they authenticate by using the NTLMSSP (NT LAN Manager Security Support Provider) provider. Data that resides on EMC Isilon clusters is unavailable to SMB/SMB2/SMB3 clients. This results in data unavailable (DU) failures. Authentication failures may also affect clients that try to access data through HTTP-based protocols such as RAN.

Workaround: Use the Kerberos protocol to authenticate Active Directory domain users.

Posted in Exchange, SharePoint, Technical, Windows | Tagged , , , , | 13 Comments

Microsoft Exchange 2010 end of mainstream support ended 1/13/14

If you are still running Exchange 2010 you now have limited support options, without an extended support contract. Microsoft will still provide security updates to it until 1/14/2020 and may provide other updates for 2010 SP3, but no guarantees on those, and only those with an Extended Hotfix Support contract are entitled to them officially.  “Mainstream” support ended on 1/13/2014, see Microsoft Support Lifecycle details on Exchange 2010 here. Pay support is still provided, but if you come across a bug/issue don’t expect a hotfix for it. The content in TechNet and other forums will still be available and is pretty solid after 5 years.

So if you’re organization is still on Exchange 2010 you need to be looking at migrating to Exchange 2013 or Office 365 ASAP!

From: http://support2.microsoft.com/gp/lifepolicy and see the Lifecycle support FAQ here.

Exchange Support

Posted in Uncategorized | 1 Comment

Remove older Exchange 2013 and IIS Log Files with this one-liner

By default Exchange 2013 logs A LOT of data in the “…\Exchange Server\V15\Logging” folder. In larger environments GBs of data can be logged under the MANY subfolders under this folder.  IIS also logs data and has since the IIS 1.0, but nowhere near as much.

A recent post on the Exchange 2013 Information Sharing Facebook group brought up this issue and a blog post on a PowerShell to script to address it. So if you want to do this via PowerShell see that post.

But the method I use is a one-liner Scheduled Task using FORFILES CLI.

Here’s the one-liner: FORFILES /p “C:\Program Files\Microsoft\Exchange Server\V15\Logging” /s /m *.* /c “cmd /c Del @path” /d -7

But you will probably want to run this daily, so follow the steps below on EACH Exchange server to remove older logs on a regular basis.

  1. Create a scheduled task with the following settings
  2. General: Runs as administrator, or an account that has delete access to the files
  3. General: Select “Run whether user is logged on or not”
  4. Trigger: Daily or how often you want it to remove older files
  5. Action: Start a program
  6. Program/script: FORFILES
    • No path needed, this EXE was included 1st in Windows Vista and is included in Windows Server 2008 and higher 
  7. Arguments: /p “C:\Program Files\Microsoft\Exchange Server\V15\Logging” /s /m *.* /c “cmd /c Del @path” /d -7
  • If you don’t have Exchange 2013 installed in the default path, change the path above to where your E15 Logging folder is
  • Change the “-7″ to control how many days to keep, “-7″ = 7 days

If you also want to delete IIS logs, create a 2nd Task to delete IIS logs, by default they are saved in “C:\inetpub\logs\LogFiles”

Here’s what the scheduled task looks like in Windows 2012 R2:
Tasks

Posted in Exchange, Technical | Tagged | 5 Comments

Exchange 2013 CU7, 2010 SP3 UR8, 2007 SP3 UR15 released & Known Issues

3/17/15 Update: 2013 CU8, 2010 SP3 UR9, & 2007 Sp3 UR16 released

12/30 Updated: Added a known issue with 2013 CU7 EDGE and two known issues from CU5 & CU6 that still exist in CU7.
12/12 Update: Exchange SP3 UR8 v2 (14.03.0224.002) released, Download here
12/10 Update #2: Exchange 2010 SP3 UR8 pulled due to MAPI RPC bug. Do NOT install UR8 v1 on Exchange 2008.
12/10 Update: 1st known issue added, for 2010 UR8

Microsoft released the latest round of updates today. As always, many updates were included and this round included security update for MS14-075, which can be downloaded for 2013 SP1/CU5 & 2013 CU6 if you just want to apply this security update and not 2013 CU7 now. This latest round was delayed due to some issues found with the install process, see Tony Redmond’s post on this here and a short post on the EHLO blog here for more info.

Microsoft Exchange Team EHLO Blog Post: Exchange releases: December 2014
Tony Redmond’s: Exchange 2013 CU7 debuts along with security fixes and updates for Exchange 2010 and Exchange 2007

2013 CU7                     Download | KB2986485
2010 SP3 UR8           Download v2 | KB2986475
2007 SP3 UR15         Download | KB2996150

Microsoft Exchange Team EHLO Blog Post: Exchange releases: December 2014

Key updates:

  • 2013: Support for Public Folder Hierarchies with up to 250,000 folders
  • 2013: Improved OAB distribution in larger environments
  • 2013: Important updates to backup
  • 2013: AD Schema update
  • 2013: Updates to backup code

If your organization is still using Public Folder and coexisting with Exchange 2007 or 2010 make sure you read this EHLO Blog post: On-Premises Legacy Public Folder Coexistence for Exchange 2013 Cumulative Update 7 and Beyond

There were some changes with the backup code\API in the 2013 CU7 update, therefore it is highly recommended that you do a full backup after installing CU7. In addition, you should test your backup AND recovery procedures and programs to ensure they are still working as expected after applying CU7. The changes affect the ESE database engine that can cause restores to fail.

Note: For those organizations running Exchange in a Hybrid configuration with Exchange Online (EXO)/Office 365 or using Exchange Online Archiving (EOA) you are required to deploy the most current updates.

This blog post will be updated to include links to other blogs and KBs as they come out about these releases. I will also add a Known Issues section once I have some items to add.

For Known Issues on 2013 CU6 see this blog post: http://blog.jasonsherry.net/2014/08/26/exchange-2013-cu6-2010-sp3-ru7-2007-sp3-ru14-released/

Known Issues

For 2013 CU7

  • Unable to install 2013 Edge role on a domain joined server
  • Microsoft Exchange Health Manager service stops
    • For more details and resolution see KB3017629
    • This issue 1st appeared in CU5
  • Safety Net hold time settings are not synced to Exchange Server 2013 Edge
    • For more details and workaround see KB3019650
    • This issue 1st appeared in CU6

For 2010 SP3 UR8

None have come to my attention yet for 2007 or 2013 updates… For Known Issues with the previous round of updates see: http://blog.jasonsherry.net/2014/08/26/exchange-2013-cu6-2010-sp3-ru7-2007-sp3-ru14-released/

Posted in Exchange, Technical | Tagged , , | 6 Comments