Wednesday, January 11, 2017

New Exchange PowerShell module with Modern Authentication support now available in Office 365 Portal

Many organizations are enabling multi-factor authentication (MFA) for all their accounts, or a subset such as for instance user accounts with an admin role. Especially with how easy this is with Office 365, even with the basic MFA feature that’s included with the license. Unfortunately enabling MFA on an admin account breaks the ability to use PowerShell to administer Exchange Online, Skype for Business Online or SharePoint.

A few months ago a new version of the Exchange PowerShell module was ‘leaked’ to the internet. It was a click-to-run executable without any documentation, but it introduced support for Modern Authentication which is a requirement for MFA.

And while there’s still no public announcement on the various Microsoft Exchange or Office blogs, not even a mention in the Office 365 Roadmap, there have been some recent updates. For starters the new PowerShell console is now available for download in the Hybrid section of the Exchange Admin Center.

image

The second is that some documentation has been published. The process was pretty self-explanatory but some official guidance is always better. The short version is this: install the application, then use Connect-EXOPSSession to create a remote session.

image

Read more in this article on the TechNet Exchange Technical Library.

Friday, December 23, 2016

Error 0xE0000100 when installing Server 2016 in a virtual machine

Today I ran into an issue while creating a new VM on a Windows Server 2016 Hyper-V host. The VM will run Server 2016 as well but throws an error message when Windows Setup loads:

image

Windows installation encountered an unexpected error. Verify that the installation sources are accessible, and restart the installation.
Error code: 0xE0000100"

I downloaded a fresh copy of the ISO, deleted and recreated the VM, rebooted but was not able to get rid of this error message. While researching I stumbled on this article: Windows Server 2012 R2 setup may fail on a virtual machine that is configured to use the minimum required memory

This article explains that this error message indicates insufficient memory. Although I assigned 1024 MB and not 512 MB I decided to add some more memory to the VM configuration.

image

To no avail, the same error occurred when I retried. Knowing now that this error indicates a low memory condition I decided to set the amount of RAM back to 1024 and disabled Dynamic Memory.  Started the VM and to my surprise, Windows Setup started and allowed me to install Windows Server 2016 on this VM. image

At this moment I’m not sure why this happened. The Hyper-V documentation for Server 2012 R2 states that in this situation, a cold boot without an OS installed, the VM should have the Startup RAM amount available. This was initially 1024 and later 1500 MB, more than the 512 MB that Windows Setup requires.

When installing or upgrading the operating system of a virtual machine, the amount of memory that is available to the virtual machine during the installation and upgrade process is the value specified as Startup RAM. Even if Dynamic Memory has been configured for the virtual machine, the virtual machine only uses the amount of memory as configured in the Startup RAM setting. Ensure the Startup-RAM value meets the minimum memory requirements of the operating system during the install or upgrade procedure.

However, in my Dynamic Memory enabled VM I had little bit less than 512 MB available.

image

Hint: Pressing shift-F10 during Windows Setup opens a command prompt, allowing you access to tools such as diskpart, chkdsk and copy. Or in this case, allowed me to query WMI.

Now in this specific situation all memory of the VM host was assigned to running VMs. Because a restarting VM requires more than the minimum configured amount of memory during boot the Hyper-V hosts uses a disk-based paging feature called Smart Paging. However, Smart Paging does not work in a cold boot situation where the VM was powered off before booting the VM.

So when these requirements are met:

  • VM with Dynamic Memory enabled
  • All host memory assigned to running VMs
  • Cold VM boot to install an operating system

The Hyper-V host is not able to make the Startup RAM amount of memory available and the VM sees the Minimum RAM amount. In this case this was 512 MB which triggered the low memory condition described in the KB article I mentioned earlier.

After freeing up some resources from this host the VM was booted again, now it showed the expected 1024 MB of memory available.

image

Server 2012 R2 or 2016?

Note that the KB article applies to Server 2012 R2 and I used the Server 2012 R2 Hyper-V documentation to learn more about memory management. From my reading there is no difference in behavior between Server 2012 R2 and Server 2016.

For reference, read:

Friday, November 4, 2016

Friendly reminder: /RecoverServer to an older OS version is not supported

In case you missed it, it’s currently not recommended to deploy Exchange 2016 on Windows Server 2016. The Windows Server team added a section dedicated to Exchange in their Windows Server 2016 Release Notes:

If you attempt to run Microsoft Exchange 2016 CU3 on Windows Server 2016, you will experience errors in the IIS host process W3WP.exe. There is no workaround at this time. You should postpone deployment of Exchange 2016 CU3 on Windows Server 2016 until a supported fix is available.

image

It’s expected that a blog post with a similar message will appear soon on the Exchange Team Blog. The difference with the first post is that this post has to include recommendations to customers that deployed Exchange 2016 on Windows Server 2016, which is supported since CU3. As the Windows Server team already stated, there is no fix or workaround available today.

To make matters worse, it looks like this issue only applies to Server 2016 servers that have been added to a DAG. Good news for customers with stand-alone Exchange 2016 on Windows Server 2016 servers but it makes recovery a tad bit more complicated.

The supported approach would be something like this:

  1. Stand up new Windows Server 2012 R2 servers
  2. Install Exchange 2016 CU3 (or CU2 to prevent this from happening)
  3. Optional: Add both servers to a DAG (not the same one, 2012 R2 and 2016 servers can’t be members of the same DAG)
  4. Move the mailboxes to the new servers
  5. Uninstall Exchange 2016 from the impacted servers

The big question here is, will the admin be able to perform all these tasks with Exchange 2016 being in this state: Exchange Server 2016 & Windows Server 2016 Datacenter IIS AppPool's constantly crashing. ECP/Powershell inaccessible

A user in a community recommended this approach:

image

There is no fix at the moment. Therefore if you have put Exchange 2016 on to Windows 2016 then the only option is to shutdown the server, rebuild as Windows 2012 R2 then do a recover server install of Exchange 2016. That works and allows you to remove it cleanly.

Unfortunately this process is not supported (*) and never has been. This can be found in Recover an Exchange Server:

What do you need to know before you begin?
The server on which recovery is being performed must be running the same operating system as the lost server. For example, you can't recover a server that was running Exchange 2013 and Windows Server 2008 R2 on a server running Windows Server 2012, or vice versa. Likewise, you can’t recover a server that was running Exchange 2013 and Windows Server 2012 on a server running Windows Server 2012 R2, or vice versa.

So deploying a new server with Server 2012 R2 and then install Exchange 2016 with the /RecoverServer option is not supported.

Let’s see what the Exchange team will have to say on this.

(*) Not supported in this context means that Microsoft Support cannot support your environment if you do this. The exception is if Microsoft Support asks you to follow this process, in that case you will be supported.

Microsoft Teams, here’s the admin guidance

Yesterday Microsoft released a preview version of Microsoft Teams, the new product to compete with Slack. It will be very interesting to see Teams integrate in Office 365 as yet another collaboration product next to Skype for Business, Exchange, Yammer, SharePoint Newsfeed and Office 365 Groups.

Currently Microsoft Teams is not enabled by default, a tenant admin has to enable the application in the Office 365 Admin Center.

image

This will change in 2017 Q1 when Teams will be enabled by default. Or as Microsoft states in the Message Center (id MC84541):

We are rolling this preview out off-by-default, to give you a chance to explore the new capabilities. This experience will be turned on-by-default in the first quarter of 2017. At this time, you can control access to this experience, at the organization level. We are working on user-level controls for this feature, and will communicate again when available.

I know that many of my customers are still struggling how to handle Office 365 Groups creation and the lack of governance and control around this area. Microsoft is working hard to improve this, as they explained in this session at Ignite 2016: Manage Microsoft Office 365 Groups But with every new Microsoft Team there will be a corresponding Office 365 Group created.

This raises the question how admins can manage the Microsoft Teams roll-out in their organization. What are the firewall requirements? And how to estimate bandwidth for peer-to-peer calling?

The good news is, there is actual admin documentation. The bad news is that it’s no longer consolidated on a single place as we’re used to with TechNet. The currently available resources for admins can be found here:

Much to my surprise the most relevant information for admins was hidden in part 2 of the MVA video series. I highly recommend to download the Deploy_Microsoft_Teams.pdf document that can be found in the Resources section of this training.

image

There’s a ton of extremely valuable information in this document that helps you understand how to prepare for Microsoft Teams, at least from an infrastructure point-of-view.

image

image

image

Have fun!

Office 365 MFA is awesome! Unless you’re an administrator…

For some reason I have never worked with MFA with Office 365 until last year. And I must say, it is awesome! Even the free version of Azure MFA that’s included with the Office 365 subscription meets the requirements of most organizations. It’s very easy to setup and configure and the end-user experience is pretty good too, supporting text messages, phone call or the Azure Authenticator app.

image

Microsoft did a great job integrating the PhoneFactor acquisition (2012) in Azure AD and Office 365. So it’s not a surprise that a lot of organizations plan to enable MFA for all users, some users or the users with an admin role. And that’s where the issue is, Office 365 MFA currently does not support Remote PowerShell. Or I should say Remote PowerShell does not offer support for MFA because this would require support for Modern Authentication. This applies not only to Exchange management, but too PowerShell management of SharePoint, Skype for Business, EOP and Security & Compliance as well.

image

How about app passwords then? We can use app passwords for applications that do not support MFA right? Unfortunately app passwords are not working either.

When talking with Microsoft Premier Support they explained there’s currently no news to share. However, a Microsoft Most Valuable Professional explored the limits of his NDA on Facebook when he disclosed that Microsoft has made a preview version of Exchange PowerShell to beta testers at the moment. I’m very keen to learn more about this new version, because currently Remote PowerShell depends on the version of PowerShell that’s installed in the OS of the workstation. I’m assuming that MFA support requires the installation of additional software.

Ironically the new Office 365 Secure Score site (https://securescore.office.com/), a challenge were organizations receive points for increasing the security, awards 50 points for organizations that enable MFA for all their Tenant Admins. There’s no mention that this removes the ability to manage Office 365 with PowerShell.

image

Keep an eye on the Office 365 Roadmap and the Azure MFA Documentation for updates.

Microsoft reverses the undocumented changed Exchange Online license removal behavior

A couple of weeks ago some users reported a change in behavior when an Exchange Online license was removed from an Azure AD user object. It was reported that with the changed behavior mail would still flow to a mailbox, after the license was removed. When a customer opened a service request with Microsoft Support, they acknowledged the change but there was no documentation available, nor was this change announced on the Office 365 Roadmap.

The first public information appeared early October when Microsoft added the following text in the License Removal section of the Delete or restore user mailboxes in Exchange Online document.

Previously, in Exchange Online, if you removed an EXO license the user was kept, but the mailbox user was transformed into a mail user and the mailbox was moved to the recycle bin. You could then recover the mailbox within the 30 days time limit.

This has now changed in Exchange Online. If you remove a user's license, the user mailbox will no longer be able to sign in and use Exchange Online or Office 365. The user mailbox will remain in Exchange Online until it is deleted, permanently removed or purged by the Office 365 admin. You can reassign a license to the user and make the mailbox active again.

But if you look in this document today the text was removed and replaced with a link to this post on the Official Exchange Team Blog: Change in behavior for delicensed Exchange Online users

image

In this post Microsoft explains the change in more detail, apologizes for the way they handled the change and most important, that they rolled back the change for now:

Due to extensive feedback from customers we are rolling back this feature to the original behavior in order to improve the feature before we release it again in an easier format (and with better documentation).

For more information, read the full post here: Change in behavior for delicensed Exchange Online users

Tuesday, October 18, 2016

Microsoft and their frequent product name changes

Today I learned that the Microsoft Federation Gateway was rebranded to Azure Authentication System some time ago. This reminded me of that one time when we made a marketing video and had the narrator use the name Windows Azure when Microsoft decided to rebrand the service to Microsoft Azure. We had to pay the voice actor again to record the text for a second time, now with Microsoft Azure.

Word Cloud-1Sometimes a new name makes perfect sense, for instance when Microsoft renamed the RPC over HTTP protocol to Outlook Anywhere. Or when they rebranded Business Productivity Online Suite (deskless worker anyone?) to Office 365.

In the Exchange space the favorite buzzword today seems to be Modern. The newly architected Public Folders in Exchange 2013 and Exchange Online became Modern Public Folders. The smoother ADAL based authentication became Modern Authentication and I overheard someone jokingly using the term Modern Outlook Anywhere for MAPI/HTTP.

Smart branding can add tremendous value to a product, but inconsistent and frequent renaming will confuse customers and hurt the recognizability of a product. An example that comes to mind is the product we know today as Skype for Business, this product received a new name with every single release. Another bad example is the recent rebranding of the Exchange web interface to Outlook on the web. I always grin when I find another ‘Outlook Web App’ in an Exchange 2016 TechNet article. It looks like even the technical writers have a hard time keeping up with the name changes. I can’t blame them.

Wednesday, October 5, 2016

Focused Inbox for Outlook is delayed

When Microsoft announced Focused Inbox for Outlook and Outlook on the Web (OWA) in July they planned to release the new features to First Release customers starting early September 2016. Roll-out to the 4th ring of customers was scheduled for October.

image

One month between First Release and GA may seem much, but for organizations that need some more time to understand the impact and communicate the changes with the end-users, a month isn’t that much time.

But more importantly, September has already passed and we have not seen the new feature to appear nor any new updates on the Office Blog or documentation for administrators.

Last week I attended Microsoft Ignite in Atlanta and had the pleasure to speak with some members of the Outlook time. My understanding is that the feature is ready, the documentation has been written but the actual deployment has been rescheduled to the November/December timeframe.

Microsoft has promised to do a better job than they did with the roll-out of Clutter. Documentation for admins should be published before launch to First Release tenants. If you want to be prepared, read up on Focused Inbox admin controls in my previous article.

Thursday, September 22, 2016

Exchange Online and the new email address limit

Exchange Online, just as any other cloud service, is a shared environment where resources are pooled between multiple tenants. This means that certain limits need to be enforced, either to ensure that the services is being used as intended or to prevent that some users consume an uneven share of the available resources.

Luckily most limits, not all, are documented quite well in the Exchange Online Limits section of the Office 365 Service Descriptions. One of the tables on that page contains some limits with regards to recipients. See the following screenshot of this table as it was on the 10th of July, 2016: (click to enlarge)

image

And here is what recently changed:

Capture

A new Recipient proxy address limit was added to the table and immediately enforced.

Interesting is that the column for Exchange 2013 is populated with the value of 200 now too:

image

Unless a recent CU introduced a hardcoded limit I don’t think this is accurate. By my knowledge the real limit in the on-premises world is the character limit of the proxyAddresses AD attribute.

Now this may not apply to you, but there are an awful lot of people out there who have up to 300 or more proxy addresses. Some users created custom addresses for each mailing list of vendor account as Exchange never implemented a wildcard email address feature (jetzemellema+amazon@gmail.com).

And to make matters worse, the admin interfaces do not allow to remove individual email addresses and then save the object again. A possible work around is to export all proxy addresses to CSV, remove them all, clean up the CSV to contain <200 entries and add them again with PowerShell.

The easiest long-term solution appears to be to add additional Distribution Groups where your mailbox is the only member. Now add a bunch of those addresses to the DG to ensure you can still receive all messages sent to the addresses.

In hindsight this would’ve been a perfect topic for Microsoft to announce before implementing the change, including guidance for customers who are impacted by this change.