DMARC broke your mailing lists!

Posted on 04/10/2014 by Jason Sherry

4/25/14 Update: Renamed post to “DMARC” instead of Yahoo now that Aol has their DMARC policy set to reject: http://postmaster-blog.aol.com/2014/04/22/aol-mail-updates-dmarc-policy-to-reject/

After spending sometime the last couple of days checking and rechecking my SPF settings I figured out today it was a change Yahoo made that broke message delivery to external recipients. Yahoo made this change “over the weekend” per some news articles, I first noticed NDRs dues to this on Monday 4/7/2014

The issue is that Yahoo changed their DMARC, which is made up for SPF and/or DKIM settings/policies, to “p=reject” which tells receiving email servers to reject emails from yahoo.com addresses that don’t originate from its servers. So if a Yahoo users sends an e-mail to yourmailinglist@yourorg.com and it contains recipients at @yahoo, @gmail, @msn, @hotmail, @outlook.com, @comcast, and many other email providers that check the original sending servers (Yahoo in the case) policy they will reject the mail. This is because the sending server of the e-mail is yourorg.com and not yahoo.com.

Yahoo did this to help reduce spam that is being sent from accounts on their servers to mailing list that contains external recipients. But they basically “broke every mailing list in the world” to quote some of the many news articles I found today about this issue.

At this time there the only workaround for Exchange clients is to use the EDGE role and setup address rewrite rules. Here is an article on Using Header Rewriting with Exchange Server 2010 that should help with that.

I’m hoping Yahoo fixes this policy setting ASAP! I will update this post as I learn more!

Errors users\DL owners will see:
mta1386.mail.bf1.yahoo.com gave this error:
Message not accepted for policy reasons. See http://postmaster.yahoo.com/errors/postmaster-28.html
mx3.hotmail.com # #SMTP#

imta13.westchester.pa.mail.comcast.net gave this error:
oFxW1n00k0D7utr0DFxXU1 Message rejected due to DMARC. Please see http://postmaster.comcast.net/smtp-error-codes.php#DM000001

BAY0-MC3-F11.Bay0.hotmail.com gave this error:
(BAY0-MC3-F11) Unfortunately, messages from (63.227.36.10) on behalf of (yahoo.com) could not be delivered due to domain owner policy restrictions.

The only workarounds that I’m aware of at this time is to use an EDGE address rewrite rule to have any messages sent to a DL to have their From address to be the DLs, instead of the sending users. The other is to enable moderation on your DLs that have external recipients and for any coming from yahoo.com or aol.com (currently, but more will be added in the future I’m sure) resent them manually from the DL or a mailbox.

I’m hoping to hear back from some of my peers soon on better workarounds.

Good blog post on DMARC and why the rejection is happening and why this is a good thing, in general: http://huitema.wordpress.com/2014/04/21/about-dmarc-or-can-email-evolve/.

Yahoo’s article on their DMARC policy and how to deal with it, but doesn’t have any info on Microsoft Exchange: http://yahoomail.tumblr.com/post/82426900353/yahoo-dmarc-policy-change-what-should-senders-do

Articles about this issue:
http://www.pcworld.com/article/2141120/yahoo-email-antispoofing-policy-breaks-mailing-lists.html
http://www.theregister.co.uk/2014/04/08/yahoo_breaks_every_mailing_list_in_the_world_says_email_guru/
http://thehackernews.com/2014/04/yahoos-new-dmarc-policy-destroys-every.html
http://www.circleid.com/posts/20140408_yahoo_addresses_a_security_problem_by_breaking_every_mailing_list/
http://www.spamresource.com/2014/04/run-email-discussion-list-heres-how-to.html

Posted in Exchange, Microsoft, Technical | Tagged , | 6 Comments

Known Issues with Exchange 2013 SP1

Posted on 03/04/2014 by Jason Sherry

5/23/14 2013 SP1 CU5 released: https://blog.jasonsherry.net/2014/05/28/exchange-2010-sp3-ur6-and-2013-cu5-released/

Last updated: 4/14/14

  1. Important: 3/4 Transport server doesn’t start after upgrade and some products that use Transport Agents don’t work
  2. Important: When building a DAG cmdlet will fail if NetBIOS and AD short domain name don’t match – 4/14/2014
    • From this FB post: https://www.facebook.com/groups/MSEX2013/permalink/801310969898022/
      • In Exchange 2013 SP1, when building a DAG, you will see issues with Set-DAG and Add/Remove-DatabaseAvailabilityGroupServer cmdlets if your Domain NETBIOS name is not equal your AD short name, eg NETBIOS name is “FOO”, and your domain name is “bar.contoso.local”. Both cmdlets will terminate with a Dr Watson. The DAG will still build fine, except for the FSW resource – there will be no FSW assigned to the DAG. We already have a FIU you can request through PSS, or create the FSW manually as a temporary workaround.
    • Work around: Manually create the FSW, contact PSS for fix, wait for public fix
  3. Minor: “Ceres” Search Foundation install error
    • From Paul Robichaux’s blog post
      • “When deploying the RTM build of Exchange 2013 SP1, I found that one of my servers was throwing an error I hadn’t seen before during installation”
  4. Minor: Default apps in Outlook Web App do not work if Exchangeis installed in Window Server 2012R2
    • From KB2938292 posted on 3/19/2014
      • “When you install Microsoft Exchange Server 2013 on a Window Server 2012 R2-based computer, default”
      • This issue will be fixed in Exchange Server 2013 Cumulative Update 5
    • Work around: Edit the web.config and add the following line to the “<appSettings> section:
      UseLegacyRequestUrlGeneration” value=”true”/>
      • 3/25: A peer of mine is working on a blog post with more details and I will post a link to it when he post it.

Related items

  1. 3/4 Outlook2013SP1 Issue: Fails to connect to Exchange in a multi-forest environment

Related Posts

Posted in Exchange, Microsoft, Technical | Tagged , , | 6 Comments

Quick post: Exchange 2013 SP1, Exchange 2010 SP3 RU5, & 2007 SP3 RU13 was just released

Posted on 02/25/2014 by Jason Sherry

It’s been awhile since I’ve posted, been busy on projects, but wanted to at least let everyone know that Exchange 2013 SP1 (aka CU4), 2010 SP3 RU5, 2007 SP3 RU13, & Office 2013 Sp1 was just released.

EHLO Post on the Exchange 2013 SP1 update 

See Tony Redmond’s blog post here for more details for now: http://windowsitpro.com/blog/exchange-2013-sp1-mixture-new-and-completed-fixtures

Downloads: Exchange 2013 SP1 | Exchange 2010 SP3 RU5 | Exchange 2007 SP3 RU13 | Office 2013 SP1

I will also create a blog post for 2013 & 2010 to talk about what’s in each of these updates and to track ‘known’ issues with them.

Big new feature in Exchange 2013 SP1, which requires Outlook 2013 SP1, is MAPI/HTTP. More to come in a later post, but for now you can view a video of Joe Warren, Exchange developer, on it here: http://channel9.msdn.com/Events/Open-Specifications-Plugfests/Redmond-Interoperability-Protocols-Plugfest-2013/Exchange-2013-and-MapiHttp

Posted in Exchange, Technical | Tagged , | Leave a comment

Got an idea for something you would like to see in Exchange?

Posted on 12/10/2013 by Jason Sherry

If so go here: http://exchange.ideascale.com/ and you can post your idea or vote on it if someone else already has one similar.

Tony Redmond blogged about this site also here: http://thoughtsofanidlemind.com/2013/12/09/exchange-improvements-site/. He is planning on having a Q&A session with Perry Clarke, Microsoft CVP for Exchange, on 12/13. For more on that see Tony’s post here: http://windowsitpro.com/blog/what-question-would-you-ask-microsofts-exchange-development-supremo

So we, the Exchange MVPs & others, have started to populate this idea list and could use your help on voting (up or down) and posting new ideas. We hope this site will be come a great place for sharing and voting on ideas and that the Microsoft Exchange team will start to look at it for ideas on what to do or fix in future releases.

Posted in Exchange | Tagged , | Leave a comment

New Exchange security updates (MS13-105) released as RUs for 2007, 2010, and SUs for 2013

Posted on 12/10/2013 by Jason Sherry

Today Microsoft is releasing security updates for Exchange 2007 SP3, 2010 SP2, 2010 SP3, 21013 CU2, & 2013 CU3 for MS13-105. For Exchange 2007 SP3 & 2010 SP2 these are included in the latest Rollup Updates. For Exchange 2013 Microsoft is following the updates plan they documented in the EHLO blog post “Servicing Exchange 2013” and will provide a Security Update (SU) package to be installed on top of CU2 or CU3.

So the new RUs for 2007 will be 2007 SP3 RU12 (KB2903911 | Download), for 2010 SP2 RU8 (KB2903903 | Download), and 2010 SP3 RU4 (KB2905616 | Download). For Exchange 2013 this update will be installed on top of CU2 or CU3, 2013 CU2 (KB2880833 | Download) and 2013 CU3 (KB2880833 | Download). An update for Exchange 2013 CU1 is NOT being provided, per the Microsoft policy of only support the current version (CU3) and one version back (CU2).

  • To summarize it another way:
      1. 2007 SP3 RU12 = 2007 SP3 RU11 + new security fixes
      2. 2010 SP2 RU8 = 2010 SP2 RU7 + new security fixes
      3. 2010 SP3 RU4 = 2010 SP3 RU3 + new security fixes
      4. Exchange 2013 CU2 will get a SU package containing the new required security fixes and the previously released security fix so you only need to apply one SU if you never applied the original one
      5. Exchange 2013 CU3 will get a SU package containing only the new required security fixes since CU3 was released
  • Issues addressed
    1. Updates Oracle OutsideIn libraries (previously known as Stellant) to a non-vulnerable version
    2. Removes a XSS attack vector in OWA logon
    3. Removes a deserialization attack vector by setting EnableViewStateMac in OWA

Mostly from the EHLO Blog post: Released: Microsoft Security Bulletin MS13-105 for Exchange

For Exchange Server 2007 & 2010, the update is being delivered via an NEW Update Rollup. UR3 will ONLY contain this security fix for MS13-105 and the other changes that were in UR2.

For Exchange Server 2013, this security updates is being delivered as discrete update and contains no other changes.  Security updates for 2013 are cumulative in nature based upon a given Cumulative Update.  This means customers who are running CU2 who have not deployed MS13-061 can move straight to the Cu3 update because it will contain both updates.  Customers who are already running MS13-061 on CU2 may install MS13-105 on top of MS13-061 without removing the previous release.  If MS13-061 was previously deployed, Add/Remove Programs will indicate that both updates are installed.  If MS13-061 was not previously deployed, only MS13-105 will appear in Add/Remove Programs.

All of these fixes will be available immediately on the download center and through Windows Update per our standard security release practice.  Note that we will not be releasing Exchange Server 2010 Service Pack 3 Update Rollup 3 to Windows Update due to the closeness of these releases and to avoid the supersedence confusion created with Update Rollups that are labeled as security releases vs. those that are not.  Windows Update will indicate that Update Rollup 4 supersedes Update Rollup 2 avoiding the problem of Windows Update offering Update Rollup 3 to customers who have Update Rollup 4 installed already.

Posted in Exchange, Technical | Tagged , , | Leave a comment

Speaking at MEC 2014 in April on cross forest migrations

Posted on 12/10/2013 by Jason Sherry

Update: Content can be viewed here: http://channel9.msdn.com/Events/MEC/2014/DMI302, but my notes, which included links to the scripts I talk about were removed. Go here: http://1drv.ms/1Gu4sfx to get the full copy of my PPT, with notes.

I just got my acceptance letter that my “Cross forest migrations: Free or 3rd party
tools?” session was accepted for MEC 2014. MEC will be in Austin, TX 3/31 – 4/2 and will have a near exclusive focus on Exchange and Office 365. Most of the speakers will be non-Microsoft, so you will get a lot of real world sessions with very little marketing spin. There will also be many of the Exchange product team people there from Microsoft also speaking.

So if you run Exchange in your environment or use Office 365 for your organization this is the conference to come to! Register now and get more details at: http://www.iammec.com/


Session Abstract:
This session will cover the tool and steps required to migrate from an Exchange 2003 or higher to Exchange 2013 in another forest. A high-level outline of the steps, scripts, Microsoft tools, and notes from the fields will be discussed. In addition, options for organizations still on Exchange 2003, which isn’t supported by built-in scripts
and tools with Exchange 2013, will be covered.

Discussion will cover the scripts included with Exchange 2013 to migrate mailboxes across forests and how to migrate contacts, groups, policies, Public Folders, and other settings that aren’t migrated by those scripts. The main focus will be the many additional steps, solutions, and scripts required to do a full fidelity migration.

This session will mainly focus on the needs of small to medium companies (< 1,000s of mailboxes). Information discussed will be helpful to any size organization that needs to do a cross forest migration.

Tags: Deployment & Migration | Public Folders | Coexistence
Audience: IT Manager/Executive | Messaging Administrator | Messaging Architect | IT Professional
Technical Level:300 – Advanced level
Product: Exchange Server 2013 | Exchange Server 2010 | Exchange Server 2007

In 2012 I also spoke at MEC, see my post here: https://blog.jasonsherry.net/2012/08/17/speaking-at-mec-2012/ and for links to my content (PPT & Word doc) goto: https://blog.jasonsherry.net/2012/09/28/mymec2012content/

Posted in Exchange, Microsoft, Technical | Tagged , , | 1 Comment

Exchange 2010 SP3 RU3 and 2013 CU3 Released & Known Issues

Posted on 11/25/2013 by Jason Sherry

3/4/14: Blog Post: Known Issues with 2013 SP1
12/4 Update: Added Known Issues list at the end

Today, after a few delays, both Exchange 2010 SP3 RU3 and Exchange 2013 RTM CU3 have been release! For more details on CU3 and a bit of a backstory on this release and the general quality issues Microsoft has had with Exchange recently see Tony Redmon’s post here: http://windowsitpro.com/blog/seeking-quality-exchange-2013-cu3

Exchange 2010 SP3 RU3: Download | KB2891587 | EHLO Blog Post
Exchange 2013 RTM CU3: Download | KB2892464 | EHLO Blog Post

I haven’t seen much about what updates are in 2010 SP3 RU3, but when I do find them I will update this blog.

For Exchange 2013 RU3 there are several key updates:

  • AD schema updates (so plan accordingly)
  • Addresses issues with 3rd party backup software, see KB2888315 for more details
  • Windows 8.1/IE11 no longer require the use of OWA Light
  • Usability improvements when adding members to new and existing groups in the Exchange Administration Console
  • Online RMS available for use by non-cloud based Exchange deployments
  • Improved admin audit log experience

The What’s New in Exchange Server 2013 and Release Notes should also be updated shortly to reflect changes in 2013 RTM CU3.

Note: Make sure you set PowerShell execution policy to “Unrestricted” before installing 2013 RU3, see KB981474 for details and steps.

Microsoft has also publicly stated that the next update to Exchange 2013 will be SP1 in early 2014 (aka as CU4). SP1 will provide Windows 2012 R2 support, S/MIME support in OWA, Edge Transport Server Role, and the various fixes and improvements expected in a SP. The Exchange Team also posted this EHLO blog: Exchange Server: The Road Ahead to dispel any myths that there won’t be an E16 or higher on-premise version.

  •  CU are basically a Service Pack that may include new features, but those features may not be documented. CU & SPs for Exchange 2013 are FULL installs, or in-place upgrades, with no un-install support. So make sure you test the CU2 BEFORE installing them in production. Microsoft is calling CU4 a “Service Pack” because this is the first time they are documenting new features and changes that were in the previous CUs. In addition, there are lifecycle requirements tied to “Service Pack” updates.

Reminder:  Customers in hybrid deployments where Exchange is deployed in-house and in the cloud, or who are using Exchange Online Archiving with their in-house Exchange deployment are required to maintain currency on Cumulative Update releases

Known Issues
Like before this is the list of issue I AM aware of, not an official list from  Microsoft. For my previous known issues with Exchange 2013 CU2 see: https://blog.jasonsherry.net/2013/07/09/exchange-2013-rtm-cu2-released/. I’ll add more details and linked to Technet Forums and KBs later this week. Comment on any issues you have come across that you are pretty sure is a bug based on others having the same issue, please include a link to TechNet forums post on the topic.

  • I need to review my CU2 issues list and check which of these issues CU3 resolves. On my to-do list 🙂
  1. Critical: 12/4 Outlook (all version) on Windows XP cannot access Exchange 2013 CU3 modern Public Folders, when their mailbox is on 2013 CU3 server
    • Only critical if you have Windows XP clients, which many organizations still do
    • Workaround: Have users use OWA or W7+ workstation
  2. Important: 12/4 Availability (Free/Busy) and OOF not working after installing KB2837618 or KB2825677 for mailboxes on Exchange 2007 in Outlook 2013
    •  The November Outlook 2013 KB2837618 security update, also in OL13 Oct update KB2825677 breaks Outlook 2013 if the user’s mailbox is on Exchange 2007
      • This issue is new to CU3
      • KB2825677 is now in Windows Update, so if you have Exchange 2007 you need to remove this update!
      • Users will see “Your automatic reply settings cannot be displayed because the server is currently unavailable. Try again later” error message when they click on their Automatic Replies (Out of Office) button.
    • Workaround: Remove updates and recreate Outlook profile, just removing the update doesn’t fix the issue
  3. Important: 12/4 Severe performance problems with IE8 and FF24 on Windows XP
  4. Important: 12/4 EAS proxy breaks after EAS App Pool crashes repeatedly
    • This is an issue with Exchange 2013 (RTM – CU3) when it is proxying EAS traffic to Exchange 2007 or 2010 based mailboxes
    • You will see Event ID 4999 | Error | MSExchange Common | With “Watson reported…” in the details
    • I’ve seen this issue at two clients, who were migrating from 2007 to 2013. In both cases users weren’t noticing any issues.
      • After migrating the last mailboxes to 2013 these errors went away
  5. Moderate: 12/4 EAS ExternalURL & InternalURLs for EAS are cleaned after installing CU3
    • This has been an issue since 2010 when doing a recovery operation and when installing 2013 CU1 and higher. I don’t recall which URLs are wiped but as a standard policy I create a “Set-URLs.ps1” script that I run after installing the CUs, and on initial setup
      • See Jeff Guillet’s post for more details on the history of this issue: bit.ly/1jmOg4l
    • Workaround: Reset them
      • CU1 & CU2 also cleared the OWA & ECP URLs, so at least those were fixed in CU3
  6. Minor: 12/4 Certificate based authentication not working
  7. Minor: 4/2/13 Exchange XML application configuration files are overwritten during cumulative update installation
    • Any customized per-server settings you make in Exchange XML application configuration files, for example, web.config files on Client Access servers or the EdgeTransport.exe.config file on Mailbox servers, will be overwritten when you install an Exchange Cumulative Update (CU). Make sure that you save this information so you can easily re-configure your server after the install. You must re-configure these settings after you install an Exchange CU.
    • From Release notes page
Posted in Exchange, Microsoft | Tagged , | 10 Comments

Exchange 2013: Transport Service & Transport Submission service fail to start fix

Posted on 11/21/2013 by Jason Sherry

Ran into this issue in my lab which started some point AFTER installing CU2. Things were working after installing CU2, but as of today the “Microsoft Exchange Transport” and “Microsoft Exchange Mailbox Transport Submission” server would not start.

On startup the error would below would occur in the Application log.

To resolve this error I did the following

  1. Opened the Exchange 2013 EAC
  2. Goto Servers\<server>
  3. Clicked DNS lookups
  4. External & Internal lookups were set to “All network adapters”
  5. Change this to “Microsoft Hyper-V Network Adapter #2” (Choose the one that Exchange is using for external\internal network traffic (NOT iSCSI or DAG one)
  6. Repeat on both External & Internal
  7. Restart the “Microsoft Exchange Transport” and “Microsoft Exchange Mailbox Transport Submission” services

Log Name: Application
Source: Application Error
Date: 11/21/2013 12:46:29 PM
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: IZSRVEX02.company.com
Description:
Faulting application name: MSExchangeSubmission.exe, version: 15.0.620.20, time stamp: 0x512192de
Faulting module name: Microsoft.Exchange.Net.ni.dll, version: 15.0.620.18, time stamp: 0x511c5c25
Exception code: 0xc00000fd
Fault offset: 0x00000000006670c7
Faulting process id: <varies>
Faulting application start time: <varies>
Faulting application path: C:\Program Files\Microsoft\Exchange Server\V15\Bin\MSExchangeSubmission.exe
Faulting module path: C:\Windows\assembly\NativeImages_v4.0.30319_64\Microsoft.E91f4adf5#\77f4457dccc969eb2fafad3d690e9f50\Microsoft.Exchange.Net.ni.dll
Report Id: 9cf3cbec-52e5-11e3-941f-00155d024403
Faulting package full name:
Faulting package-relative application ID:

Posted in Exchange, Technical | Tagged , | 5 Comments

WTF… Blog posts from Sept & Oct missing…

Posted on 11/06/2013 by Jason Sherry

Just logged into my blog today and noticed my posts from Sept & Oct are missing. So if you are looking for one of my recent post I’m not sure where they went but I’ll see if I can find a copy and re-post them. Wonder if WordPress installed a bad update 😉

Posted in Uncategorized | Leave a comment

RIP Microsoft MCA\MCM certification programs

Posted on 08/31/2013 by Jason Sherry

See this blog post: Are Microsoft Losing Friends and Alienating IT Pros? by Steve Goodman, or this one Retiring the Microsoft Master certifications and training by Neil Johnson (who 1st broke the news I think), this one Microsoft Certified Systems Master certification now dead by Paul Robichaux, this one Microsoft is retiring the MCSM/MCA Program by Michael Van Horenbeeck, this one Ain’t Nobody [at Microsoft Learning] Got Time For That by Devin Ganger, this one by Once an MCM always an MCM by Jeff Guillet, this one Microsoft Advanced Certification (MCA, MCSM, MCM) – the end of an era by Wictor Wilén, and the many others out there starting today.

All of these bloggers are Exchange MVPs, MCA\MCMs, or MCA\MCMs in another area and are posing about Microsoft sending out an e-mail to the MCA list last night around 11PM EST announcing that they stopping the certified master\architect program on 10/1/2013. They basically canceled the program with not notice on a Friday night at 11PM on a long weekend. I’m guess they were hoping not to get much attention by doing it this way. Well they were sorely mistaken, as you can see from the multiple blog post above.

I have been an Exchange MVP now for seven years, my first year I was an MVP over MIIS (Anyone recall that product, which became to ILM, and then FIM?). We have seen the writing on the wall for awhile and Microsoft even told us that they are pushing hard to get organizations into the cloud 5+ years ago. Well this is another step in that process. Basically they are killing off the program that they use to train the top external and internal (about 50% were Microsoft employees) experts on Exchange, SharePoint, Directory Services (AD), and SQL. By doing this they are removing a key skill set from the community for current and future releases of these products.

I had hoped to become an MCA on Exchange, but the training is 2 weeks (used to be 3) and cost $18,000 (used to be $25K). So I wanted to make sure I was 100% ready before committing to this, there is a 70% fail rate. Besides the high cost there is also the cost of being out of work for really weeks (have to study HARD be ready and to past the final exam\lab test), flights, and hotel. So in reality this was a $30K+ commitment to make, so this is why I never got mine and I guess never will.

Last month they announced the dropping of TechNet subscription, which has been THE key source of Microsoft software for IT Pros to run in their labs. I always get my Microsoft software from here, I still will have access to the software (at least for now) via an MSDN subscription as part of being an MVP.

We, the MVPs, fear that the next step will be the cancellation of the MVP program. After all, why do you need independent external experts if Microsoft is only focus on the cloud and getting people migrated to the cloud (Office 365).

We’re pissed and it’s a bad time to be a Microsoft IT Pro.  Guess I’ll start looking to get back into Product Management or take an IT directory\CIO position. It will be a sad day when I have to take one of these jobs, I’ve been working for myself for the last 3+ years straight [almost 100% remotely] and on & off for the last 15 has been great. But I foresee me getting my frequent flier status back and wasting a lot of time traveling and being away from my family and the great state of CO.

Posted in Microsoft, Personal, Technical | Tagged , | 5 Comments