Thursday, September 22, 2016

Exchange Online and the new email address limit

Exchange Online, just as any other cloud service, is a shared environment where resources are pooled between multiple tenants. This means that certain limits need to be enforced, either to ensure that the services is being used as intended or to prevent that some users consume an uneven share of the available resources.

Luckily most limits, not all, are documented quite well in the Exchange Online Limits section of the Office 365 Service Descriptions. One of the tables on that page contains some limits with regards to recipients. See the following screenshot of this table as it was on the 10th of July, 2016: (click to enlarge)

image

And here is what recently changed:

Capture

A new Recipient proxy address limit was added to the table and immediately enforced.

Interesting is that the column for Exchange 2013 is populated with the value of 200 now too:

image

Unless a recent CU introduced a hardcoded limit I don’t think this is accurate. By my knowledge the real limit in the on-premises world is the character limit of the proxyAddresses AD attribute.

Now this may not apply to you, but there are an awful lot of people out there who have up to 300 or more proxy addresses. Some users created custom addresses for each mailing list of vendor account as Exchange never implemented a wildcard email address feature (jetzemellema+amazon@gmail.com).

And to make matters worse, the admin interfaces do not allow to remove individual email addresses and then save the object again. A possible work around is to export all proxy addresses to CSV, remove them all, clean up the CSV to contain <200 entries and add them again with PowerShell.

The easiest long-term solution appears to be to add additional Distribution Groups where your mailbox is the only member. Now add a bunch of those addresses to the DG to ensure you can still receive all messages sent to the addresses.

In hindsight this would’ve been a perfect topic for Microsoft to announce before implementing the change, including guidance for customers who are impacted by this change.

Make your HCW experience even more fun

Ever wondered what happens when you click through the new and shiny Hybrid Configuration Wizard? Wouldn’t it be awesome to be able to see what happens when you wait in real time? Now you can.

image

People with a background in Unix or Linux are probably familiar with the tail program. tail reads the output of a file and keeps doing so when the file is updated with new data. This is an ideal tool to view log files in real time.

PowerShell offers similar functionality in Get-Content with the -Wait switch. With that in mind, all we need to do is find the most recent log file in the directory as every instance of the HCW creates a new log file and then read the contents of that file.

Start the HCW first, we need the log file to be there before we can read it, and then enter the following one-liner in PowerShell:

Get-Item "$ENV:appdata\Microsoft\Exchange Hybrid Configuration\*.log" | Sort LastWriteTime | Select-Object -Last 1 | Get-Content -Wait

You like that? Then try using Get-Content -Wait against C:\ExchangeSetupLogs\ExchangeSetup.log the next time you’re installing or upgrading Exchange. Have fun!

Sunday, September 11, 2016

How to fix ALT+S in Firefox

Key combination ALT and S is commonly used to save or submit data in web application, such as Exact Online, phpBB and vBulletin. Since Mozilla Firefox 2.0 this is not working anymore.

This can be fixed by editing two settings in the advanced settings of Firefox.

  • In Firefox, visit about:config
  • Change ui.key.chromeAccess to 5
  • Change ui.key.contentAccess to 4

image

The changes are immediately effective, no need to close and reopen the application.

Friday, August 26, 2016

Update, fixed: KB3176934 breaks remote PowerShell

Update: This issue has been fixed in the re-released KB3176938 update.

Today I ran into an error message on one of my systems. PowerShell was unable to import my remote session to Exchange Online.

image

Import-PSSession : Could not load type 'System.Management.Automation.SecuritySupport' from assembly 'System.Management.Automation, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

A quick Google search learned that this KB3176934 update was released a couple of days ago and is known to break DSC, remote PS and probably other stuff.

Microsoft scheduled an updated update to be released on the 30th of August 2016. If you can’t wait for some reason, for instance you planned to do some actual work today, uninstall the update and reboot your system.

wusa /uninstall /kb:3176934

Source: PowerShell DSC and implicit remoting broken in KB3176934

Tuesday, August 16, 2016

Focused Inbox admin controls appear in Exchange Online

Back in 2014 Microsoft acquired Acompli, a company that had developed the popular mobile apps with a feature called Focused Inbox. A server side algorithm was used to “learn” the difference between important email and less important email, providing the users a very clean view of their mailbox showing only the most relevant messages.

The Acompli apps have then been rebranded to Outlook Mobile and the algorithm was migrated to Office 365 and Azure’s machine learning capabilities. The next step is to bring Focused Inbox to Outlook and Outlook on the Web, which Microsoft recently announced. See Outlook helps you focus on what matters to you.

I’m sure that any Exchange Online admin remembers how Clutter was introduced, a new and potentially confusing mailbox feature without any admin controls. With Focused Inbox Microsoft is planning to do a better job and has announced admin control before the actual roll-out to the Office 365 tenants.

image

Admins will be able to disable or enable Focused Inbox on the tenant level with Set-OrganizationConfig and the -FocusedInboxOn parameter. Similar to Clutter there will be cmdlets to manage the feature per mailbox as well, expect something like Get-FocusedInbox and Set-FocusedInbox.

Focused Inbox will begin to roll-out in the September-October timeframe, starting with First Release customers. More information on admin controls will be available before roll-out, giving admins more time to develop a strategy on how to handle the implementation of this new feature.

Monday, July 4, 2016

Outlook 2013 June 2016 update causes Mail applet to stop working

Many issues with opening the Mail applet in the Control Panel have been reported in the technical communities recently. Apparently this applies to Office 2013 Click-to-run (C2R) installs with the most recent ‘June 2016’ update installed. The build number of the affected installs is 15.0.4833.1001 and newer.

Microsoft is aware of the issue and will have this fixed in the upcoming July 2016 update. While the issue prevents the user from opening the Mail applet in Control Panel, there are several workarounds to access the Outlook profile settings to either select a different or create a new profile, or to open the Control Panel applet to edit existing profile settings.

Method 1

Start Outlook with the /profiles switch:

Outlook.exe /profiles

Method 2

Toggle the ‘Prompt for a profile to be used’ setting with a registry key.

HKEY_CURRENT_USER\Software\Microsoft\Exchange\Client\Options\PickLogonProfile

Value type = REG_SZ
1 = "Prompt for a profile to be used"
0 = "Always use this profile"

Method 3

Add two missing registry keys:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Control Panel\Cpls]
"MLCFG32.CPL"="C:\\Program Files\\Microsoft Office 15\\root\\office15\\MLCFG32.CPL"

[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Control Panel\Cpls]
"MLCFG32.CPL"="C:\\Program Files\\Microsoft Office 15\\root\\office15\\MLCFG32.CPL"

Method 4

Revert the Office 2013 C2R install to 15.0.4823.1004, the May 2016 update.

  1. Exit all Office applications.
  2. Open an elevated command prompt. To do this, click Start, type cmd in the Start Search box, right-click Command Prompt or cmd.exe, and then click Run as administrator.
  3. At the command prompt, type the following command, depending on your bitness of Windows, and then press Enter:
    For an Office 2013 installation and a 32-bit version of Windows:
    cd %programfiles%\Microsoft Office 15\ClientX86
    For an Office 2013 installation and a 64-bit version of Windows:
    cd %programfiles%\Microsoft Office 15\ClientX64
  4. Type the following command, and then press Enter:
    officec2rclient.exe /update user updatetoversion=15.0.4823.1004
  5. When the repair dialog box appears, click Online Repair.
  6. Click Repair, and then click Repair again.
  7. After the repair is complete, start Outlook.
  8. Click File, and then click Office Account.
  9. In the Product Information column, click Update Options, and then click Disable Updates.
    Important note This step is very important. The repair process re-enables automatic updates. To prevent the newest version of Office Click-to-Run from being automatically reinstalled, make sure that you follow this step.
  10. Set a reminder in your calendar for a future date to check this Knowledge Base article (3175861) for a resolution for this issue. Enable automatic updates in Office again after this issue is fixed. Enabling automatic updates again will make sure that you don't miss future updates.

KB article in the making

The instructions in Method 4 mention a KB article with id 3175861. Unfortunately that article was announced but has not been published yet.

image

To learn more about this issue I recommend to check https://support.microsoft.com/en-us/kb/3175861 in a few days to read more.

Wednesday, May 25, 2016

Multiple Transport Rule conditions and the OR operator

Here’s something I ran into today and would like to share. Exchange transport rules, also known as mail flow rules, can have multiple conditions, actions and/or exceptions which makes them flexible and a powerful tool. However, if you add multiple conditions an AND operator will be applied. This means that the rule will be triggered only when all conditions are True.

How can we replace the AND with an OR? For instance, if we want to apply a certain action when the sender is member of a group or a specific person? The answer is that we can’t do this with a single transport rule. There is an easy solution, simply create a copy of the transport rule and update the condition. Now the action will be applied when either of the transport rules is triggered because the single condition is True.

Tuesday, May 17, 2016

Exchange 2016 courses on MVA, edX and their quality

Yesterday Tony Redmond published an article titled Virtual academies, odd questions, and MCSE recertification. In the post he shows numerous examples of bad worded questions and incorrect or outdated answers on Microsoft’s MVA platform. The Exchange 2013 and Exchange Online content on MVA could definitely use a thorough upgrade.

On May the 3rd the Exchange Team announced new Exchange 2016 material: Exchange Server 2016 Online Training Courses Now Available! Most notable was that the four courses were presented of the edX platform instead of their own MVA, not at least because the edX courses have cost $ 49 each.

Today I walked through the first course: Microsoft Exchange Server 2016 - 1: Infrastructure, which is free as long as you don’t require a certificate, to get an idea of the quality. My first impression is that the quality is not the worst I’ve ever seen, but there is a lot to improve. First let’s take a look at the first two modules and check for factual errors. Make sure to continue reading because there is more…

Module 1: Exchange Server 2016 Prerequisites and Requirements

image

This information seems to be taken from the Exchange 2007 documentation: Planning Processor Configurations. Both the 1.000 mailboxes per CPU core as well as the Average profile of 10 messages sent and 40 received are from the Exchange 2007 timeframe.

 

image

The Exchange 2016 sizing guidance refers to the article for Exchange 2013. There we can read that the per mailbox memory requirements for the 50 and 100 messages profile are 12 and 24 MB, not 3 and 6 MB as stated in the course.

 

image

This command is going to fail because of the dot after -Restart.

 

image

By al means, do not install any version of WMF later than 4.0. Recently WMF 5.0 was released but this new version is currently not supported with any version of Exchange. An no, the asterisk does not refer to anything.

 

image

This command is going to fail because of the space after RSAT.

 

image

Now this is an interesting question, the answer is ‘hidden’ in the title of the question.

Module 2: Exchange Server 2016 Deployment

image

The UM role was integrated with the Mailbox server role beginning with Exchange 2013, not 2016.

 

image

Single-server recommended to run in a VM? I fully agree, but never heard this recommendation form the Exchange team. And replicate the VM to another Hyper-V server? Hyper-V Replica is NOT supported for Exchange.

 

image

It’s not, by default there’s a V15 folder in that path under where Exchange is installed.

 

image

This command will fail because the /mode switch is missing.

 

image

The correct answer is EdgeTransport, no space between the words.

 

image

The correct name was Forefront Online Protection for Exchange (FOPE). I said was, because FOPE was replaced with Exchange Online Protection (EOP) a couple of years ago. Forefront Online Protection was never the name of a product or service.

Due to time constraints I decided to stop after the first two modules.

But wait, they are on MVA too!

Initially I wanted to explain how odd it is that Microsoft used the edX platform instead of their own MVA. But when researching for this article today I discovered that the exact same courses have been published on MVA just yesterday. And when I say ‘exact same courses’, I mean the same content but now presented in a video of two people reading the same course.

image

Different format, same content and same errors (WMF 4.0 or later):

image

For me personally this format of video learning does not work at all, because the pace is too slow. I prefer to read on my own pace and be able to skip some content when I’m already familiar with a topic. But if the video format works for you, use the MVA ones and save $ 49 per course.

In conclusion

The majority of the content in the first two modules of the first course was copy and pasted from the TechNet Library and did not add any value for experienced Exchange administrators. Paid courses in a better format are on edX, the free version is on MVA as a video. Pick one that works for you.

Be aware that the learning content contains errors and more authoritative information on the topics can be found in the TechNet Library as well on the Exchange Team Blog. As the guidance and features change with every CU or Exchange Team blog post, expect the quality of the learning content to get worse over time.

Sunday, May 15, 2016

The new HCW on Exchange 2010, a few notes

Today I used the new Exchange 2010 Hybrid Configuration Notes in a production environment and wanted to share my notes. This is not an extensive review of the new HCW, just a few short remarks.

First of all, Exchange 2010 Update Rollup 13 replaces the button to open the old HCW in EMC with a link to the download page for the new HCW. If you’re not ready for the new HCW and want to do additional testing, do not upgrade the CAS server where you’d execute the HCW yet to UR13.

The new HCW requires .Net Framework 4.5 which is typically not installed on an Exchange 2010 server because Exchange 2010 uses version 3.5. Make sure the latest updates are installed after installing 4.5 on the server.

My contacts at Microsoft assured me that the new HCW would operate just as the old one did, but better. Testing discovered that this is not entirely true. The new HCW creates Send Connectors and Organization Relationships with different names than the old HCW did. If pre HCW and post HCW scripts are being used to correct the shortcomings of the HCW they need to be updated to use the new names that now contain a GUID. Common tasks after running the HCW are changing the -TargetOwaURL parameter of the Organization Relationship or update the Send Connector to use one or more Edge Subscriptions instead of an HT server.

Knipsel

The page to edit the Hybrid Domains has improved a lot. Unfortunately it’s not possible to sort on enabled status or domain name by clicking on the column header. This makes locating a domain very hard, especially when you’re managing a couple of hundred accepted domains.

The new log file is much more verbose, but you won’t find it in the most logical places. The new location is $ENV:appdata\Microsoft\Exchange Hybrid Configuration. Tip: search for the string *ERROR* or WARNING. That’s correct, the string ERROR is enclosed by double quotes, WARNING is not.

Leaving feedback is much easier with the Give feedback link on every page of the wizard. Unfortunately the HCW freezes for some minutes after sending the feedback, but be patient and the HCW can be continued.

Friday, May 6, 2016

Office Online Server released, confusion around sizing

In case you missed it, the Office team is in the process of releasing the RTM version of Office Online Server (OOS) to the public. Customers with a Volume Licensing account can download OOS from the Volume License Servicing Center, OOS will be available on MSDN beginning May 9th, 2016.

For most Exchange admins OOS as well as the previous versions of the same product, are a new technology. For a great overview of deploying Exchange 2016 with OOS I recommend to view the recording or at least the slides of the session that Michel de Rooij recently presented on this subject.

Unfortunately the documentation for OOS is not (yet) of the high standard we’re seeing with Exchange and some other products. In this post I want to highlight two topics as an example: sizing requirements and virtualization support.

Sizing your OOS servers

Maybe the comparison with Exchange is not the best example here, because Exchange 2010 was the last version where sizing documentation was of a very high quality. For recent versions of Exchange the guidance is shifting towards using the calculator to design your environment, instead of using the calculator to validate your design.

The guidance for OOS is even worse:

image

That’s odd, SharePoint 2016 is a very different application and the recommended production architecture is to spread the roles over multiple servers. SharePoint does know the Single-Server farm concept but this is recommended for development, testing or very limited production use. The SharePoint teams gives two sets of minimum requirements, one for development and one for pilot or user acceptance scenario’s:

image

We’re sizing our production OOS deployment so let’s pick the largest one: 4 CPU cores and 24 GB of memory. The assumption here is that the Office team had the SharePoint Single-Server deployment in mind when they referred to SharePoint sizing for OOS.

But wait, there is another authoritative source: the Exchange team! In the Exchange 2016 Preferred Architecture is a short section dedicated to designing your OOS servers.

image

So without asking any questions about the number of users, % of OotW usage or whether we need view-only or editing capabilities we’re now at 8 CPU cores and 32 GB of memory, times two per datacenter of course because the PA assumes HA. Please note that the SharePoint team recommends to use at least double of your memory as the free disk space, so that would make 64 GB instead of 40.

With the current lack of real-world performance figures it probably would make sense to start with a relatively small server, monitor your deployment carefully and add resources if necessary. Which brings me to my next point.

Virtualization

Just as every other modern application OOS supports deployment in a virtualized environment, giving customers the choice and flexibility to deploy OOS on their own terms.

image

The first bullet is probably good advice for performance and manageability reasons, the second bullet is basic common sense. The interesting part is hidden in the first paragraph:

…is supported when you deploy it using Windows Server Hyper-V technology…

Is Microsoft really saying that you’re allowed to deploy OOS on Hyper-V but not on VMware, Xen, KVM or any other hypervisor solution that is certified through the Windows Server Virtualization Validation Program (SVVP)? Yes they are, but this has to be a mistake. I cannot think of any valid reason behind this statement.

But wait, there is more…

While researching this subject I noticed several other interesting or questionable statements in the OOS documentation on TechNet. To name a few:

The Office team recommends SSL offloading, that means that the load balancer would be the endpoint for the SSL tunnel and that all traffic between the load balancer and the real servers will be unencrypted. This goes against the security principle of treating both external as well as internal networks as unsafe by default. It’s considered best practice to deploy SSL bridging instead. The Office team acknowledges this and recommends to mitigate the risks involved by recommending the use of firewalls and private subnets to secure the traffic.

The load balancing section mentions a requirement for layer 7 routing and client affinity but lacks any recommendations on what affinity options to choose and does not mention how to configure the load balancer’s health checks. In practice we see that a lack on guidance in this area generally leads to bad implementations.

In conclusion

I could go on for a while, but I won’t. I recommend every Exchange organization considering OOS with Exchange 2016 to perform a cost-benefit analysis to start with, for instance if 95% of the users will use non-OotW clients to access Exchange 2016 mailboxes an OOS deployment maybe doesn’t make sense. And there is of course the licensing aspect, as editing capabilities are not free and are coupled to Office suit licensing.

I you are planning your OOS deployment with Exchange 2016, make sure to contact your Microsoft representative to confirm that OOS on your hypervisor will be supported. From a sizing perspective, start with a small VM and add resources when necessary. And make sure to keep an eye on the Twitter an Blog-o-sphere for more updates on this subject.