Difference between revisions of "Exchangetoox"

(Introduction)
(Synced mail folders are not seen on Open-Xchange)
 
(36 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
== Introduction ==
 
== Introduction ==
  
The document describes a way how mailboxes can be migrated from MS Exchange to a cyrus IMAP server using imapsync without the need to know the users passwords.
+
The document describes a way how mailboxes can be migrated from MS Exchange™ to a cyrus IMAP server using imapsync without the need to know the user passwords.
  
== Pre requirements ==
+
== Requirements ==
  
# admin account for cyrus
+
# A cyrus IMAP server and an admin account
#: the admin accounts for cyrus are defined in the parameter "admins:" in the configuration file "/etc/imapd.conf". IMAP login for that user must be possible. In this examples, the admin user name is "cyrus" and it's password "cpass".
+
#: the admin accounts for cyrus are defined in the '''admins: '''  parameter of the '''/etc/imapd.conf ''' configuration file. In the examples, the admin user name is '''cyrus''' with password '''cpass'''.
# Microsoft Exchange 2003
+
# Microsoft Exchange 2003™
#:others might work, too. 2003 was the version this howto was written with.
+
#:other Microsoft Exchange™ versions might work, too. This howto was written for the 2003 version.
# postmaster account for MS Exchange
+
# IMAP connector installed and activated for Microsoft Exchange™
#:the exchange account for migration needs to have the rights to access all mailboxes on the Exchange server. In the examples the user name is "migration" with password "mpass". The user can be created following these steps:
+
# Postmaster account for MS Exchange™
## create a new "DWORD" in "HKEY_Current_User\Software\ Microsoft\Exchange\ExAdmin" with the name "ShowSecurityPage" and set this to "1".
+
#:The Exchange™ account for migration needs to have the permission to access all mailboxes on the MS Exchange™ server. In the examples the user name is '''migration''' with password '''mpass'''. The user can be created following these steps:
## open the Active Directory User and Computer console and create a new user with the name "migration", password "mpass" and a exchange account
+
## Create a new '''DWORD''' in '''HKEY_Current_User\Software\ Microsoft\Exchange\ExAdmin''' with the name '''ShowSecurityPage''' and set this to '''1'''.
## reopen the Exchange System Manger and give that new user full access to the entire mail organization (right click on the most upper entry naming your organization -> properties -> security -> add)
+
## Open the Active Directory User and Computer console and create a new user with the name '''migration''', password '''mpass''' and MS Exchange™ attributes.
 +
## Reopen the MS Exchange™ System Manager and give that new user full access to the entire mail organization (right click on the uppermost entry naming your organization -> properties -> security -> add)
  
 
== Testing the admin users ==
 
== Testing the admin users ==
  
Those two sessions descibe a telnet session where a succesfull IMAP connections gets created and closed (cyrus is runnin on localhost and the Exchange server on 10.20.30.170)
+
Those example IMAP logins describe a telnet session where a succesful IMAP connection is created and closed (cyrus is running on localhost and the Exchange server on 10.20.30.170)
 +
 
 +
=== Test if the cyrus admin can access the server ===
  
 
  []# telnet localhost imap
 
  []# telnet localhost imap
Line 32: Line 35:
 
  []#
 
  []#
  
 +
=== Test if the migration user can log in to own mailbox ===
 
   []# telnet 10.20.30.170 imap
 
   []# telnet 10.20.30.170 imap
 
  Trying 10.20.30.170...
 
  Trying 10.20.30.170...
Line 44: Line 48:
 
  Connection closed by foreign host.
 
  Connection closed by foreign host.
 
  []#
 
  []#
 +
 +
=== Test if the migration user can log in for user test in domain ===
 +
 +
This tests shows if it is possible to access the mail box "test" with the credentials of the migration user:
 +
 +
[]# telnet 10.20.30.170 imap
 +
Trying 10.20.30.170...
 +
Connected to mastercane.open-xchange.com (10.20.30.170).
 +
Escape character is '^]'.
 +
* OK Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0 (win2003.perf.open-xchange.local), steht zur Verf�gung.
 +
. login domain/migration/test mpass
 +
. OK LOGIN completed.
 +
. logout
 +
* BYE Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0, meldet sich ab.
 +
. OK LOGOUT completed.
 +
Connection closed by foreign host.
 +
[]#
 +
 +
== Migrating a single user ==
 +
 +
Access to the MS Exchange™ mailbox is done via '''<domain>/migration/<user to migrate>''' and the password of the migration account. In cyrus the Data is stored as user with the credentials of the admin account. The imapsync command is as follows:
 +
 +
imapsync --host1 <ms exchange> --user1 <domain>/migration/<user to migrtate1> --password1 \
 +
          mpass --host2 <cyrus server> --user2 <user to migrate2> --authuser2 cyrus \
 +
          --password2 cpass --authmech2 PLAIN --prefix1 "INBOX." --regextrans2 \
 +
          's/INBOX.INBOX/INBOX/'  --noauthmd5 --subscribe \
 +
          --exclude "&ANY-ffentliche Ordner|Aufgaben|Journal|Kalender|Kontakte|Notizen" --dry
 +
 +
{| border="1" cellspacing="0" cellpadding="5" align="center"
 +
! variable
 +
! value
 +
|-
 +
| <ms exchange>
 +
| hostname or ip of the Exchange server
 +
|-
 +
| <domain>
 +
| Active directory domain name
 +
|-
 +
| <user to migrtate1>
 +
| Exchange user to be migrated
 +
|-
 +
| <cyrus server>
 +
| hostname or ip of the cyrus imap server
 +
|-
 +
| user to migrate2
 +
| target user name in the cyrus imap server
 +
|-
 +
| --exclude
 +
| a list of system folders not to migrate
 +
|-
 +
|}
 +
 +
The above command does actually do a try run without migrating any data. This is because of the '''--dry''' option. It is needed to see what special folders are available that do not need to be migrated. These are the folders calendar,contacts,tasks,notices and shared folder. They differ from each other in the single languages. It is also save to check if the standard folders are the same on both sides (if not, that can be tuned with (multiple are possible) --regextrans2 and a regular expression that fixes that).
 +
 +
Removing the '''--dry''' option will actually migrate the user.
 +
 +
Then, the sync can run multiple times e. g., a comprehensive job can run to migrate the majority of the data and once again just before the user changes to the new server.
 +
 +
== Bulk import of multiple users ==
 +
 +
=== Sample for importing OX users out of the OX database ===
 +
 +
This script basically lists all user from a Open-Xchange context and uses the user id as username for the imap servers on both sides to export and to import all.
 +
 +
#!/bin/sh
 +
 +
# Target OX context id
 +
CID=1
 +
 +
# cyrus settings
 +
CYRHOST=localhost
 +
CYRUSADM=cadm
 +
CYRPWD=cpass
 +
 +
# exchange settings
 +
MSHOST=10.20.30.170
 +
MSDOM=domain
 +
MSADM=madm
 +
MSPASS=secret
 +
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"
 +
 +
 +
export PATH=$PATH:/opt/open-xchange/sbin/
 +
 +
 +
for user in $(listuser -c $CID | grep -v ^Id | awk '{ print $2 }'); do
 +
 +
        imapsync --host1 "$MSHOST" \
 +
                --user1 "${MSDOM}/${MSADM}/${user}" \
 +
                --password1 "$MSPASS" \
 +
                --host2 "$CYRHOST" \
 +
                --user2 "$user" \
 +
                --authuser2 "$CYRUSADM" \
 +
                --password2 "$CYRPWD" \
 +
                --authmech2 PLAIN \
 +
                --prefix1 "INBOX." \
 +
                --regextrans2 's/INBOX.INBOX/INBOX/' \
 +
                --noauthmd5 \
 +
                --subscribe \
 +
                --exclude \""$SYSFOLD"\" \
 +
                --dry
 +
 +
done
 +
 +
remove '''--dry''' so the script actually takes action.
 +
 +
=== Sample for importing LDAP users out of Active Directory ===
 +
 +
This script queries a Active Directory server for all "sAMAccountName" attributes in the users tree to use it as IMAP usernames for export and import them all.
 +
 +
#!:/bin/sh
 +
 +
# cyrus settings
 +
CYRHOST=localhost
 +
CYRUSADM=test1111
 +
CYRPWD=S3cr3t3
 +
 +
# exchange settings
 +
MSHOST=10.20.30.170
 +
MSDOM=perf
 +
MSADM=migration
 +
MSPASS=secret
 +
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"
 +
MSDN="CN=Administrator,CN=users,dc=perf,dc=open-xchange,dc=local"
 +
MSDNPW="netline"
 +
MSIGNORE="^#|SUPPORT|WIN2003|krbtgt|Administrator|Gast"
 +
 +
export PATH=$PATH:/opt/open-xchange/sbin/
 +
 +
for user in $(ldapsearch -x -D${MSDN} -w "$MSDNPW" '(objectclass=user)' sAMAccountName | grep sAMAccountName | grep -v -E "($MSIGNORE)" | awk '{ print $2 }'); do
 +
 +
        imapsync --host1 "$MSHOST" \
 +
                --user1 "${MSDOM}/${MSADM}/${user}" \
 +
                --password1 "$MSPASS" \
 +
                --host2 "$CYRHOST" \
 +
                --user2 "$user" \
 +
                --authuser2 "$CYRUSADM" \
 +
                --password2 "$CYRPWD" \
 +
                --authmech2 PLAIN \
 +
                --prefix1 "INBOX." \
 +
                --regextrans2 's/INBOX.INBOX/INBOX/' \
 +
                --noauthmd5 \
 +
                --subscribe \
 +
                --exclude \""$SYSFOLD"\" \
 +
                --dry
 +
 +
done
 +
 +
remove '''--dry''' so the script actually takes action.
 +
 +
== Known issues ==
 +
 +
=== Synced mail folders are not seen on Open-Xchange ===
 +
 +
#Verify if the missing mail folders show up in the output of imapsync and if that is not the case tune the '''--exclude''' option
 +
#Verify if the missing folders are there but not subscibed ( right click on '''INBOX''' -> '''subscribe folders''' in the Open-Xchange web gui). It is possible to set '''com.openexchange.mail.ignoreSubscription=true''' in '''"/opt/open-xchange/etc/groupware/mail.properties"''' to let all folders show up regardless of their subscription status.

Latest revision as of 07:23, 1 September 2009

Introduction

The document describes a way how mailboxes can be migrated from MS Exchange™ to a cyrus IMAP server using imapsync without the need to know the user passwords.

Requirements

  1. A cyrus IMAP server and an admin account
    the admin accounts for cyrus are defined in the admins: parameter of the /etc/imapd.conf configuration file. In the examples, the admin user name is cyrus with password cpass.
  2. Microsoft Exchange 2003™
    other Microsoft Exchange™ versions might work, too. This howto was written for the 2003 version.
  3. IMAP connector installed and activated for Microsoft Exchange™
  4. Postmaster account for MS Exchange™
    The Exchange™ account for migration needs to have the permission to access all mailboxes on the MS Exchange™ server. In the examples the user name is migration with password mpass. The user can be created following these steps:
    1. Create a new DWORD in HKEY_Current_User\Software\ Microsoft\Exchange\ExAdmin with the name ShowSecurityPage and set this to 1.
    2. Open the Active Directory User and Computer console and create a new user with the name migration, password mpass and MS Exchange™ attributes.
    3. Reopen the MS Exchange™ System Manager and give that new user full access to the entire mail organization (right click on the uppermost entry naming your organization -> properties -> security -> add)

Testing the admin users

Those example IMAP logins describe a telnet session where a succesful IMAP connection is created and closed (cyrus is running on localhost and the Exchange server on 10.20.30.170)

Test if the cyrus admin can access the server

[]# telnet localhost imap
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
* OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS AUTH=LOGIN AUTH=PLAIN SASL-IR] servera Cyrus IMAP4 v2.3.7-Invoca-RPM-OX-2.3.7-3.OX server ready
. login cyrus cpass
. OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH] User logged in
. logout
* BYE LOGOUT received
. OK Completed
Connection closed by foreign host.
[]#

Test if the migration user can log in to own mailbox

 []# telnet 10.20.30.170 imap
Trying 10.20.30.170...
Connected to mastercane.open-xchange.com (10.20.30.170).
Escape character is '^]'.
* OK Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0 (win2003.perf.open-xchange.local), steht zur Verf�gung.
. login migration mpass
. OK LOGIN completed.
. logout
* BYE Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0, meldet sich ab.
. OK LOGOUT completed.
Connection closed by foreign host.
[]#

Test if the migration user can log in for user test in domain

This tests shows if it is possible to access the mail box "test" with the credentials of the migration user:

[]# telnet 10.20.30.170 imap
Trying 10.20.30.170...
Connected to mastercane.open-xchange.com (10.20.30.170).
Escape character is '^]'.
* OK Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0 (win2003.perf.open-xchange.local), steht zur Verf�gung.
. login domain/migration/test mpass
. OK LOGIN completed.
. logout
* BYE Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0, meldet sich ab.
. OK LOGOUT completed.
Connection closed by foreign host.
[]#

Migrating a single user

Access to the MS Exchange™ mailbox is done via <domain>/migration/<user to migrate> and the password of the migration account. In cyrus the Data is stored as user with the credentials of the admin account. The imapsync command is as follows:

imapsync --host1 <ms exchange> --user1 <domain>/migration/<user to migrtate1> --password1 \
         mpass --host2 <cyrus server> --user2 <user to migrate2> --authuser2 cyrus \
         --password2 cpass --authmech2 PLAIN --prefix1 "INBOX." --regextrans2 \
         's/INBOX.INBOX/INBOX/'  --noauthmd5 --subscribe \
         --exclude "&ANY-ffentliche Ordner|Aufgaben|Journal|Kalender|Kontakte|Notizen" --dry
variable value
<ms exchange> hostname or ip of the Exchange server
<domain> Active directory domain name
<user to migrtate1> Exchange user to be migrated
<cyrus server> hostname or ip of the cyrus imap server
user to migrate2 target user name in the cyrus imap server
--exclude a list of system folders not to migrate

The above command does actually do a try run without migrating any data. This is because of the --dry option. It is needed to see what special folders are available that do not need to be migrated. These are the folders calendar,contacts,tasks,notices and shared folder. They differ from each other in the single languages. It is also save to check if the standard folders are the same on both sides (if not, that can be tuned with (multiple are possible) --regextrans2 and a regular expression that fixes that).

Removing the --dry option will actually migrate the user.

Then, the sync can run multiple times e. g., a comprehensive job can run to migrate the majority of the data and once again just before the user changes to the new server.

Bulk import of multiple users

Sample for importing OX users out of the OX database

This script basically lists all user from a Open-Xchange context and uses the user id as username for the imap servers on both sides to export and to import all.

#!/bin/sh

# Target OX context id
CID=1

# cyrus settings
CYRHOST=localhost
CYRUSADM=cadm
CYRPWD=cpass

# exchange settings
MSHOST=10.20.30.170
MSDOM=domain
MSADM=madm
MSPASS=secret
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"


export PATH=$PATH:/opt/open-xchange/sbin/


for user in $(listuser -c $CID | grep -v ^Id | awk '{ print $2 }'); do

        imapsync --host1 "$MSHOST" \
                --user1 "${MSDOM}/${MSADM}/${user}" \
                --password1 "$MSPASS" \
                --host2 "$CYRHOST" \
                --user2 "$user" \
                --authuser2 "$CYRUSADM" \
                --password2 "$CYRPWD" \
                --authmech2 PLAIN \
                --prefix1 "INBOX." \
                --regextrans2 's/INBOX.INBOX/INBOX/' \
                --noauthmd5 \
                --subscribe \
                --exclude \""$SYSFOLD"\" \
                --dry

done

remove --dry so the script actually takes action.

Sample for importing LDAP users out of Active Directory

This script queries a Active Directory server for all "sAMAccountName" attributes in the users tree to use it as IMAP usernames for export and import them all.

#!:/bin/sh

# cyrus settings
CYRHOST=localhost
CYRUSADM=test1111
CYRPWD=S3cr3t3

# exchange settings
MSHOST=10.20.30.170
MSDOM=perf
MSADM=migration
MSPASS=secret
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"
MSDN="CN=Administrator,CN=users,dc=perf,dc=open-xchange,dc=local"
MSDNPW="netline"
MSIGNORE="^#|SUPPORT|WIN2003|krbtgt|Administrator|Gast"

export PATH=$PATH:/opt/open-xchange/sbin/

for user in $(ldapsearch -x -D${MSDN} -w "$MSDNPW" '(objectclass=user)' sAMAccountName | grep sAMAccountName | grep -v -E "($MSIGNORE)" | awk '{ print $2 }'); do

       imapsync --host1 "$MSHOST" \
               --user1 "${MSDOM}/${MSADM}/${user}" \
               --password1 "$MSPASS" \
               --host2 "$CYRHOST" \
               --user2 "$user" \
               --authuser2 "$CYRUSADM" \
               --password2 "$CYRPWD" \
               --authmech2 PLAIN \
               --prefix1 "INBOX." \
               --regextrans2 's/INBOX.INBOX/INBOX/' \
               --noauthmd5 \
               --subscribe \
               --exclude \""$SYSFOLD"\" \
               --dry

done

remove --dry so the script actually takes action.

Known issues

Synced mail folders are not seen on Open-Xchange

  1. Verify if the missing mail folders show up in the output of imapsync and if that is not the case tune the --exclude option
  2. Verify if the missing folders are there but not subscibed ( right click on INBOX -> subscribe folders in the Open-Xchange web gui). It is possible to set com.openexchange.mail.ignoreSubscription=true in "/opt/open-xchange/etc/groupware/mail.properties" to let all folders show up regardless of their subscription status.