$Id: FAQ.Exchange.txt,v 1.81 2024/01/11 14:24:49 gilles Exp gilles $ This documentation is also available online at https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt ====================================================================== Imapsync tips for Exchange 2000 2003 2007 2010 2013 2016 2019 ====================================================================== Questions answered in this FAQ are: Q. Can I use imapsync to transfer from or to Exchange accounts? Q. How to sync from XXX to Exchange 2010/2013/2016/2019 Q. I see "NO Maximum size of appendable message has been exceeded" What can I do with that? Q. Exchange throttles the sync and says: "Request is throttled. Suggested Backoff Time: 299961 milliseconds". What can I do with that? Q. For Exchange I have double and triple checked the username and password spelling but I still get a "LOGIN failed". Any clue? Q. Exchange fails with "User is authenticated but not connected". Q. How can I access a shared mailbox? Q. How can I use a shared account as a backup account for several mailboxes and so avoid spending too many dollars in backup accounts? Q. Exchange fails with "BAD Command received in Invalid state". Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says "could not append ... NO Server Unavailable. 15" Q. From XXX to Exchange 2013, read receipts are all resent again after a sync. Even for old messages. How can I fix that? Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error Q. From XXX to Exchange 2010/2013 I get this error message sometimes: "BAD Command Argument Error 11". What does it mean? Q. From XXX to Exchange 2010 or 2013, the flag Flagged does not seem to be well synced. What can I do? Q. How to migrate from or to Exchange 2007/2010/2013 with an admin/authuser account? Q. How to migrate from or to Exchange 2016 with an admin/authuser account? Q. How to migrate from or to Exchange 2003 with an admin/authuser account? Q. Couldn't create folder [trash] "Mailbox already exists". Q. Migrating to Exchange 201O, messages get date of the transfer, this is bad for sorting and listing. What can I do? Q. How to sync from any to Exchange 2007? Q. Exchange 2007 folders ending with a space cause an imapsync crash. "Could not select: socket closed while reading data from server" Q. How to sync from Microsoft Exchange 2000 IMAP4rev1 server? Now the questions again with their answers. ====================================================================== Q. Can I use imapsync to transfer from or to Exchange accounts? R. Yes. But IMAP access to an Exchange account is not always allowed by default so it has to be allowed in the server configuration part. Exchange: https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help ====================================================================== Q. How to sync from XXX to Exchange 2010/2013/2016/2019 R0. IMAP is not enable by default on Exchange, see how to enable it: https://docs.microsoft.com/en-us/exchange/enable-imap4-in-exchange-2013-exchange-2013-help Also read http://clintboessen.blogspot.com/2018/03/binding-certificate-breaks-imap-or-pop.html R1. Following is a command line resume that solves most encountered issues when migrating to Exchange. To fully understand or change the details you have to read the next Q/R sections. On Windows, use: imapsync.exe ... --exchange2 which is equivalent in imapsync release 1.970 to: imapsync.exe ... ^ --maxsize 10_000_000 ^ --maxmessagespersecond 4 ^ --disarmreadreceipts ^ --regexflag "s/\\Flagged//g" ^ --regexmess "s,(.{10239}),$1\r\n,g" On Unix, use: imapsync ... --exchange2 which is equivalent in imapsync release 1.970 to: imapsync ... \ --maxsize 10_000_000 \ --maxmessagespersecond 4 \ --maxlinelength 10239 \ --regexflag 's/\\Flagged//g' \ --disarmreadreceipts \ --regexmess 's,(.{10239}),$1\r\n,g' On Linux, you can also try the "reformime" command that can be used like: imapsync ... --maxlinelengthcmd "reformime -r7" To get reformime, install the "maildrop" package. On Linux again, there is a good Python script in the tarball that can fix several things that Exchange or O365 have issues with. Use it like this: ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py It often does some miracles on messages. ====================================================================== Q. For Exchange I have double and triple checked the username and password spelling but I still get a "LOGIN failed". Any clue? R1. Disable double-step authentication, also known as 2-factor, 2-step authentication on the Azure/Active Directory portal. R2. Microsoft introduced something called "security defaults" which is enabled by default for new tenants. One of the rules blocks IMAP access as of imapsync. The funny thing is that you can't disable single rules of this security package without buying additional licenses. Switching the whole thing off allows the login. Also, disable double-step authentication on the Azure/Active Directory portal. See here: https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/concept-fundamentals-security-defaults Thanks to Stephan Buhre for this R2 answer. R3. I want to migrate to Exchange 2019. I haven't been able to sync using admin credentials even if I've tried and made all the configuration mentioned in the FAQ.Exchange. The error was always "LOGIN failed". My problem was that I have been used an admin account which haven’t attached a mailbox in Exchange. This is my admin account which I use only for configuration in Exchange, not for emails. So, after I’ve created in Exchange a mailbox for this user, the sync started to work. I can confirm that imapsync is working for Exchange 2019, with plain authentication, and full permissions of admin account (this account must have mailbox attached) applied to regular user mailbox account. It doesn't work with the --authuser2 parameter. The syntax that works for me is: imapsync ... --host2 192.168.xxx.yyy --user2 domain.local\admin.account\regular.user --password2 "xxxxxxx" or imapsync ...--host2 192.168.xxx.yyy --user2 admin.account@mycompany.com\regular.user --password2 "xxxxxxx" Thanks to Ioan Nechita for this R3 answer. R4. Are there special characters in the password? https://imapsync.lamiral.info/FAQ.d/FAQ.Authentication_failure.txt https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Windows.txt https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Unix.txt https://imapsync.lamiral.info/FAQ.d/FAQ.Passwords_on_Mac.txt ====================================================================== Q. Exchange fails with "User is authenticated but not connected". R1. "The message User is authenticated but not connected is due to a bug in the Exchange server's IMAP implementation. If the client presents a valid user name but an invalid password, the server accepts the login, but subsequent commands fail with the aforementioned error message." Source: https://unix.stackexchange.com/questions/164823/user-is-authenticated-but-not-connected-after-changing-my-exchange-password Thanks to James Abbottsmith for this link and explanation at https://github.com/imapsync/imapsync/issues/32#issuecomment-153561647 R2. Miguel Alameda reported understanding and solving this issue like this, the context was admin/authuser: "The admin user had not permission in the target mailbox." R3. Jason from magiclamp.net adds this tip (2021_11_23). "In GoDaddy Exchange admin center, under recipients, you can double click on a mailbox, and in the "mailbox delegation" scrolling down to the bottom it shows a input field named "Full Access". Add the admin account there." ====================================================================== Q. How can I access a shared mailbox? R. First, create a shared mailbox, for example shared@example.com. Then give full permissions to a licensed account user@example.com. Now with imapsync, use the licensed user login with the syntax: user@example.com\shared@example.com and the password for user@example.com Caveat: Character \ is a special character, so use double-quotes around the user login, like: imapsync ... --user1 "user@example.com\shared@example.com" Sources: https://adam-hand.com/2017/07/25/connect-a-shared-mailbox-from-o365-to-outlook-via-imap/ https://www.arclab.com/en/kb/email/imap-settings-shared-mailbox-office-exchange-online.html https://social.technet.microsoft.com/Forums/en-US/336e02ee-6767-4810-90a0-1352bd7cc9e9/office-365-how-to-access-a-shared-mailbox-using-imap-client?forum=onlineservicesexchange ====================================================================== Q. How can I use a shared account as a backup account for several mailboxes and so avoid spending too many dollars in backup accounts? R. Use the option --subfolder2 backup_foo imapsync ... --user2 sharedloginsyntax --subfolder2 backup_foo To restore, do the reverse with: imapsync ... --user1 sharedloginsyntax --subfolder1 backup_foo ====================================================================== Q. I see "NO Maximum size of appendable message has been exceeded" What can I do with that? R1. With Exchange 2019 Change the recipient Receive/Send size: Set-mailbox -Identity user@contoso.com -MaxReceiveSize 100mb -MaxSendSize 100mb Thanks to Pasternak Denis for this Exchange 2019 tip. R2. With Exchange 2016 Thanks to Torsten Bergemann from Germany: In order to have exchange accept big e-mails via imap, several size limitations have to be removed on several levels of the exchange software: For the organization (within exchange admin panel), for each connector (with exchange admin panel), for each Postfach (using exchange management shell). For details and instructions, check https://docs.microsoft.com/en-us/Exchange/mail-flow/message-size-limits?view=exchserver-2019 https://docs.microsoft.com/en-us/Exchange/architecture/client-access/client-message-size-limits?view=exchserver-2019 Unfortunately, the second link does not give instructions to remove the limit from imap4 service. To solve it for Exchange 2013/2016, add the line to the file C:\Program Files\Microsoft\Exchange Server\V14 [or V15]\ClientAccess\PopImap\Microsoft.Exchange.Imap4.exe.config Please note that this change may be lost each time you install an cumulative update. After the change, restart the services exchange transport role, exchange information store and exchange imap4 Backend. Source in Deutch: https://social.technet.microsoft.com/Forums/de-DE/2c07cf43-fe0e-4d22-b061-676a67c6d58d/exchange-2013-max-mailgre-per-imap?forum=exchange_serverde This change made our Exchange installation accept bigger e-mails in order to remove the imapsync Exchange error message "NO Maximum size of appendable message has been exceeded" ====================================================================== Q. Exchange throttles the sync and says: "Request is throttled. Suggested Backoff Time: 299961 milliseconds". What can I do with that? R0. Exchange have throttle mechanisms to limit any huge usage. Sometimes imapsync transfers are too stressful for servers. The message "Request is throttled. Suggested Backoff Time: 299961 milliseconds" comes from the imap Office365 server, imapsync just reports it before being disconnected from it. R1. Play with options --maxbytespersecond or --maxmessagespersecond or --exitwhenover imapsync ... --maxbytespersecond 100_000 imapsync ... --maxmessagespersecond 2 imapsync ... --exitwhenover 1_000_000_000 I don't know the upper value that avoid the default throttling from 0365 and I guess it changes over time. R2. For Exchange, in case throttle appears anyway, fix them with: https://docs.microsoft.com/en-us/exchange/change-user-throttling-settings-for-specific-users-exchange-2013-help See also: http://linux-france.tk/prj/imapsync_list/msg02072.html (Subject: Re: [imapsync] From Exchange to Dovecot. Speed Date: Mon, 03 Nov 2014 00:14:59 +0100) Sometimes restarting the Exchange server is needed to take into account the change in the configuration. ====================================================================== Q. Exchange fails with "BAD Command received in Invalid state". R1. This message might happen when authenticating without ssl nor tls, wich is rare now since imapsync tries to do ssl then tls by default. Add --tls1 or else --ssl1 if this error message comes from host1. Add --tls2 or else --ssl2 if this error message comes from host2. R2. This message might also happen when authenticating with ssl and tls. Don't use --tlsX and --sslX on the same side X (1 or 2), since it asks for a double encryption, an encryption inside an encryption. It won't work, you can't do a direct double encryption, few servers allow that. ====================================================================== Q. Exchange fails with "NO Server Unavailable. 15" and imapsync says "could not append ... NO Server Unavailable. 15" R1. The problem could be a mailbox quota limitation. In the Exchange log file, if you see many "Microsoft.Exchange.Data.Storage.QuotaExceededException", it looks like "Quota Exceeded" could be the issue. The solution is to increase the quota for this mailbox. Quoting https://docs.microsoft.com/en-us/exchange/recipients/user-mailboxes/storage-quotas?view=exchserver-2019 In the Exchange admin center (EAC), navigate to Recipients > Mailboxes. In the list of user mailboxes, click the mailbox that you want to change the storage quotas for, and then click Edit icon, then change "Prohibit send and receive at (GB)", then save your changes. Or use the Exchange Management Shell, aka PowerShell: Set-Mailbox -Identity "Joe Healy" -IssueWarningQuota 499gb -ProhibitSendQuota 500gb -ProhibitSendReceiveQuota 500gb -UseDatabaseQuotaDefaults $false R2. A user fixed it by enabling IMAP logging on Exchange which is quite crazy as a solution. But why not, here it is: https://www.mantisbt.org/forums/viewtopic.php?p=68835&sid=8b1ecd2d976e85218fbf33fcd38c209d#p68835 by TerranceM Set-ImapSettings -Server -ProtocolLogEnabled $true ====================================================================== Q. From XXX to Exchange 2013, read receipts are all resent again after a sync. Even for old messages. How can I fix that? R. imapsync can remove the header containing this read receipt request. On Unix or Windows use: imapsync ... --disarmreadreceipts Since read receipts should be sent for unseen messages that will go to a seen state after the migration, you could be strict and apply the regex only to seen messages. Selecting seen message can be done with: imapsync ... --search1 "SEEN" --disarmreadreceipts Of course a second run has to be run without the --disarmreadreceipts for unseen messages: imapsync ... --search1 "UNSEEN" If fact --disarmreadreceipts is just an option equivalent to: --regexmess 's{\A(.*?(?! ^$))^Disposition-Notification-To:(.*?)$}{$1X-Disposition-Notification-To:$2}igxms' That regex changes the header Disposition-Notification-To. It prefixes it with an X- so that it becomes inactive. Disposition-Notification-To: blabla becomes X-Disposition-Notification-To: blabla Thanks to David Karnowski for pointing and solving this issue. ====================================================================== Q. DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error DEBUG: IO/Socket/SSL.pm:1043: local error: SSL read error R1. "SSL read or write error" happens sometimes, it isn't related to imapsync directly but to the ssl underlying library when communicating with Exchange in TLS/SSL encrypted mode. Next runs should put the sync further, so rerun the syncs until it is well completed. R2. Another solution is to remove --tls or --ssl options for Exchange and accept a clear text sync. R3. See also the FAQ FAQ.SSL_errors.txt https://imapsync.lamiral.info/FAQ.d/FAQ.SSL_errors.txt ====================================================================== Q. From XXX to Exchange 2010/2013 I get this error message sometimes: "BAD Command Argument Error 11". What does it mean? R. This error message comes from Exchange IMAP server when it encounters any problem. Most of the time it is one of the following: * Some messages are bigger than the size limit. 10 MB by default on Exchange. It can be upped by configuration for Exchange. Sometimes a reboot is needed to take configuration changes into account. If you can't fix this limit on Exchange then use option --maxsize 10000000 for 10 MB, change it if needed) to tell imapsync to skip those messages. imapsync ... --maxsize 10000000 # 10 MB for Exchange * Quota reached. The whole account is full. It can be upped by configuration. * Some messages have some lines too long. Use option --maxlinelength to skip messages whose max line length is over a number of bytes. --maxlinelength 1000 is a RFC2822 must but most server support higher values. Exchange supports 9900 characters line length: imapsync ... --maxlinelength 9900 In case you prefer fixing messages with long lines the hard way, instead of skipping them with --maxlinelength 9900, just use: On Windows imapsync ... --regexmess "s,(.{9900}),$1\r\n,g" On Unix imapsync ... --regexmess 's,(.{9900}),$1\r\n,g' Have also in mind that Exchange closes the connection after 10 errors encountered so you might also see "BYE Connection closed" errors from Exchange, which means Exchange leaves the session and say goodbye, come back later. Rerun a sync then. On Linux, there is a good Python script in the tarball that can fix several things that Exchange have issues with. Use it like this: ./imapsync ... --pipemess W/tools/fix_email_for_exchange.py It often does some miracles on messages. ====================================================================== Q. From XXX to Exchange 2010 or 2013, the flag Flagged does not seem to be well synced. What can I do? R. Use the following trick. Run imapsync twice, one with --regexflag and one without, like this: 1) imapsync ... --regexflag "s/\\Flagged//g" 2) imapsync ... You can add --debugflags if you want to see what imapsync gets and does in details with flags. The magic of this trick is on ignoring the \Flagged flag on the first sync and setting it on the second sync, with STORE instead of APPEND. This Exchange bug seems that Exchange gets and sets well the Flagged flag with APPEND in IMAP but then it forgets it with other protocols; With STORE it sets and gets the "\Flagged" flag everywhere. Thanks to Dave Murray and Simon Savva for reporting and solving this issue. ====================================================================== Q. How to migrate from or to Exchange 2007/2010/2013 with an admin/authuser account? R. The tricks comes from Michele Marcionelli and Benjamin Priestman: This may works: imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ... or imapsync ... --user2 "domain\admin2\user2" --password2 adminpassword2 ... or imapsync ... --user2 "admin2@domain\user2" --password2 adminpassword2 ... where "domain" is set be the user's UPN in Active Directory or the NETBIOS or DNS name of the domain. The exact format might vary depending on local configuration and you should experiment with the different formats. PLAIN authentication is the only way to go with --authuser1 for now. So don't use --authmech1 SOMETHING with --authuser1 admin_user, it will not work. Same behavior with the --authuser2 option. A little note from Michael Scherer. The previous workaround in the FAQ seems to be obsolete. I can confirm that imapsync ... --host2 exchange_server \ --authuser2 admin@domain --user2 user_alias is working without any glitches, running Exchange 2010 SP2 here. Explanation, Exchange knows a) about a username and b) about a mailNickname which can be different to the username. As it seems you need to use the mailNickname (e.g. alias) defined for user2, not the username itself. https://techcommunity.microsoft.com/t5/exchange-team-blog/understanding-login-strings-with-pop3-imap/ba-p/610683 https://web.archive.org/web/20191014050543/https://techcommunity.microsoft.com/t5/Exchange-Team-Blog/Understanding-login-strings-with-POP3-IMAP/ba-p/610683 the second link has the same content, I give it in case the first disappears, thanks to archive.org! A little note from John Becker. After you pointed out that I had issues with authorization, I realized that as an admin user I don't have all the rights on the mailboxes. So I added myself to the user mailbox with full rights. I also changed the login type on the Exchange CAS to type 1, plain text. The command that I used for imapsync is: imapsync ... ^ --host2 exchange-server ^ --user2 "windows-domain\admin-user\aaaa" ^ --password2 admin-password ... ====================================================================== Q. How to migrate from or to Exchange 2016 with an admin/authuser account? R. This might work: imapsync ... --user2 user2 --authuser2 admin2 --password2 adminpassword2 ... Note from https://github.com/imapsync/imapsync/issues/136#issuecomment-397020248 The key is to not use the built-in Administrator account which Microsoft seems to have blocked access to in imap. I'm doing a sync to Exchange 2016 accounts and this works: imapsync ... --ssl2 --user2 account@domain.com ^ --authuser2 accountwfullaccess@domain.com ^ --password2 "accountwfullaccesspw" The admin account you use needs to have full access to the account(s) you want to sync which you can setup in the gui or via powershell and again - don't use administrator@domain.com - as that will never work because IMAP LOGIN is disabled for that account apparently under all circumstances. Note from Noxyron https://github.com/imapsync/imapsync/issues/136#issuecomment-543761463 Hi guys. For a long time I could not start synchronization, I use admin rights and the --authuser2 switch when I migrated from Mdaemon to Exchange2016. I always got the error: "Host2 failure: Error login on [my.exchange.com] with user [mydomainuser] auth [PLAIN]: 2 NO AUTHENTICATE failed." In order for ImapSync to login to the user's mailbox using administrator rights, you need to give for the administrator the permissions to the mailbox of this user with the command: Add-MailboxPermission -Identity DomainUser -User DomainAdmin -AccessRights FullAccess -InheritanceType All -AutoMapping $false using Exchange Management Shell. Without this, synchronization will not work. Please, add this comment to the FAQ.Exchange.txt (done!) ====================================================================== Q. How to migrate from or to Exchange 2003 with an admin/authuser account? R. Thomas Edgar wrote the following In case you can glean something from this snippet which allowed us to migrate from Exchange 2003 t0 Cyrus Imap 2.4 (you would reverse the flow to go the other way): imapsync --dry --host1 ExchangeServer.mycompany.com \ --user1 Some.User@mycompany.com --authuser1 ExchangeAdminAccount \ --proxyauth1 --password1 '$pass_with_dollars$' \ --host2 cyrusimapbackend.mycompany.com \ --user2 SameOrDiffererentUser@mycompany.com \ --authuser2 CyrusAdminAccount --password2 CyrusAdminPassword We also needed to: - Ensure the Exchange Admin Acct had IMAP4 enabled in it's profile (it initially didn't!) - Ensure the Some.User Exchange Acct had IMAP4 enabled in it's profile (it initially didn't!) - Add the CyrusAdminAccount to admins line in /etc/imapd.conf - Give CyrusAdminAccount lrswipkxtecda to the Cyrus Imap account being migrated to (- or in your case, from) In case you are not aware: - It will prompt for a password if you don't supply it - the domain part of the fully-qualified email could be omitted in our case - Use --debugimap when testing initial connectivity, if necessary ====================================================================== Q. Couldn't create folder [trash] "Mailbox already exists". R. Some servers take care about character case in folder names, some servers do not, like Exchange. Since non-respecting case can merge two different folders into one then imapsync respects case. For example, if a host1 server has a folder name called "trash" and the host2 server already has a folder "Trash" or "TRASH" then imapsync will try to create the folder "trash" on host2, because trash and Trash are different strings. But if host2 does not respect character case it will consider folder "trash" already exists and will say it, that's the error message reported by imapsync: "Mailbox already exists", message coming from the server. The folder creation fails but messages are well transferred in so take a look at this warning, understand why it happens and it should be fine most of the time. To avoid this warning use --regextrans2 to map the folder names imapsync ... --regextrans2 "s/^trash$/Trash/" If there are two folders Trash and trash on host1 then both will be merge into only one Trash folder on host2. In case option --delete2 is used the regextans2 above becomes mandatory, otherwise imapsync will sync messages from the first Trash and then delete them when syncing trash. In order to avoid merging folders that are considered different on host1 but the same on destination host2 because of case sensitivities and insensitivities, use --nomixfolders ====================================================================== Q. Migrating to Exchange 201O, messages get date of the transfer, this is bad for sorting and listing. What can I do? R1. Be sure to have at least Exchange 2010 SP2 Rollup 5 https://tribalchicken.net/imap-sync-issues-with-exchange-2010/ R2. See also the Outlook side (thanks to Martin Hochreiter for this solution) by changing Archive to sent/receive date https://www.howto-outlook.com/faq/archivenotworking.htm https://web.archive.org/web/20190914063435/http://www.howto-outlook.com/faq/archivenotworking.htm It's often shorter to change one server than thousands clients so R1 might be easier to do. ====================================================================== Q. How to sync from any to Exchange 2007? Several problems: - Big messages: increase the "send- and receive-connector" in exchange2007 to 40 MB. R. 2 solutions R1. With imapsync (only partial success) --skipheader 'Received' \ --regexmess 's{\A(.*?(?!^$))^Date:(.*?)$}{$1Date:$2\nReceived: From; $2}gxms' Any user having time to spend to debug Exchange 2007 with imapsync is welcome. R2. Other solution Two users succeeded by using "MS Transporter Suite" (which is closed expensive non-free software). ====================================================================== Q. Exchange 2007 folders ending with a space cause an imapsync crash. "Could not select: socket closed while reading data from server" R. Using another imap tool like Thunderbird or Outlook, change the folder name to exclude the trailing space. See https://github.com/imapsync/imapsync/issues/154 ====================================================================== Q. How to sync from Microsoft Exchange 2000 IMAP4rev1 server? R. imapsync ... --prefix1 "INBOX." ====================================================================== ======================================================================