$Id: FAQ.Use_cache.txt,v 1.11 2021/04/22 13:20:18 gilles Exp gilles $ This document is also available online at https://imapsync.lamiral.info/FAQ.d/ https://imapsync.lamiral.info/FAQ.d/FAQ.Use_cache.txt ======================================================================= Imapsync --usecache or --useuid options ======================================================================= Questions answered in this FAQ are: Q. What is --usecache good for? Q. What is the technical principle of --usecache? Q. On Windows, with --useuid and after a reboot or a new login, a new run generates duplicates. What can I do to avoid and fix that? Q. On Windows, with --useuid or --usecache a problem occurs with long nested folder names. The error message is: "No such file or directory; The filename or extension is too long" Q. Inode problem with --usecache or --useuid on Linux Now the questions again and their answers. ======================================================================= Q. What is --usecache good for? R. The option --usecache is useful to speed up next syncs with --usecache. The first sync with --usecache between two imap accounts is as low as the first sync without --usecache, the next syncs are faster. ======================================================================= Q. What is the technical principle of --usecache? R. Option --usecache creates a empty file per email message in order to keep the UIDs mapping between account1 and account2. The cache is maintened at each run. Messages UIDs enregistred in the cache are not fetched again to be identified. It really speeds up the process of syncing. The cache directory is /imapsync_cache/ where is the temporary directory given by the --tmpdir option or the one by default, /tmp on Unix, variable on Windows. ======================================================================= Q. On Windows, with --useuid and after a reboot or a new login, a new run generates duplicates. What can I do to avoid and fix that? R. The cache is build in a temporary directory. Imapsync uses a system call to get this temporary directory, it is not hardcoded and it is system dependent, the call garanties a write access. Sometimes on Windows this directory name changes upon reboot or for any other reason. Anyway, the temporary directory used by imapsync can be set by the option --tmpdir and that's the way to avoid this issue. The new problem is that the temporary directory must be writable, which is not a big deal to found. Use for example either a) b) or c) a) The current directory: imapsync ... --tmpdir . b) The root directory: imapsync ... --tmpdir / c) Any directory you want, like: imapsync ... --tmpdir "C:\TEMP" Now, to fix the duplicates on host2, have a run with the option --delete2duplicates imapsync ... --delete2duplicates ======================================================================= Q. On Windows, with --useuid or --usecache a problem occurs with long nested folder names. The error message is: "No such file or directory; The filename or extension is too long" R. This comes from a Windows limitation on pathnames. No more than 260 characters are allowed for pathnames. See more details on page http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247.aspx#maxpath The workaround solution given at the previous link, ie using \\?\D:", does not work for imapsync. So this imapsync Windows bug is still pending and needs a fix using a different technique to cache, like a database file for example. A solution to fix the issue is to use a Linux virtual host on a Windows box, with VirtualBox or VmWare etc. There is no bug this way, pathnames can be several thousands characters long. Better said that done but not so difficult nor expensive these days, VirtualBox is free and VmWare Player is free for personal or test use. If you have to stick on Windows, there are two good workarounds to reduce the cache directory name: 1) Use --tmpdir "D:\\temp" or simply --tmpdir "D:" and imapsync will build and use the cache in the sub directory D:\imapsync_cache\ 2) add two equivalent entries in the etc/hosts for host1 imap.truc.org and host2 imap.trac.org. If you map the ip of imap.truc.org just with the letter a and same thing for imap.trac.org then you gain characters etc/hosts 192.168.12.1 a 192.168.55.3 b Then use: imapsync --host1 a --host2 b ... You can get the ip of a host with the ping command line C:\> ping imap.truc.org 3) A third solution is to not use options --useuid nor --usecache Fixing this long path problem directly in imapsync is in the TODO file for a very long time. ======================================================================= Q. Inode problem with --usecache or --useuid on Linux R. You may run out of inodes using --usecache, especially with large migration. Option --usecache creates a empty file per email message in order to keep the UIDs mapping between account1 and account2. So, if you plan to sync regularly 10 millions messages over a period of migration then the filesystem of --tmpdir needs 10 millions of free inodes. If it hasn't those free inodes then create a new special filesystem devoted to the imapsync cache. # Create a file of 10 GB (10 millions*1024): dd if=/dev/zero of=/var/tmp/fscache bs=1M count=10000 # Create a filesystem where each file is only 1024 bytes per inode: mkfs.ext2 -F -i 1024 /var/tmp/fscache # Mount this brand new filesystem mkdir -p /var/tmp/cachedir mount -o loop /var/tmp/fscache /var/tmp/cachedir df -i /var/tmp/cachedir # count inodes available # Tell imapsync to use it imapsync ... --tmpdir /var/tmp/cachedir/ # When the migration is over and the cache becomes really useless umount /var/tmp/cachedir/ rm /var/tmp/fscache ======================================================================= =======================================================================