http://opendedupe.org
Opendedup develops SDFS, a Deduplication File-System for Linux.
The main features of SDFS are:
- Reduced Storage Utilization – SDFS Deduplication can reduce storage utilization by up to 90%-95%
- Scalability – SDFS can dedup a Petabyte or more of data. Over 3TB per gig of memory at 128k chunk size.
- Speed – SDFS can perform deduplication/redup at line speed 290 MB/S+
- VMWare support – Work with vms – can dedup at 4k block sized. This is required to dedup Virtual Machines effectively
- Flexible storage – deduplicated data can be stored locally, on the network across multiple nodes, or in the cloud.
- Inline and Batch Mode deduplication – The file system can dedup inline or periodically based on needs. This can be changed on the fly
- File and Folder Snapshot support – Support for file or folder level snapshots.
# cat /etc/issue
Ubuntu 10.04.1 LTS \n \l
# uname -a
Linux unicorn 2.6.32-24-server #39-Ubuntu SMP Wed Jul 28 06:21:40 UTC 2010 x86_64 GNU/Linux
# sudo -s
# cd /usr/local/src
# wget http://sources.silverice.org/jdk/7-ea-bin-b97-linux-x64-10_jun_2010/jdk-7-ea-bin-b97-linux-x64-10_jun_2010.bin
# chmod +x jdk-7-ea-bin-b97-linux-x64-10_jun_2010.bin
# ./jdk-7-ea-bin-b97-linux-x64-10_jun_2010.bin
# export JAVA_HOME=/usr/local/src/jdk1.7.0
# apt-get install libselinux1-dev
# wget http://opendedup.googlecode.com/files/debian-fuse.tar.gz
# tar zxf debian-fuse.tar.gz
# cd debian-fuse
# dpkg –install *.deb
# wget http://opendedup.googlecode.com/files/sdfs-latest.tar.gz
# tar zxf sdfs-latest.tar.gz
# cd sdfs-bin
# ./mkfs.sdfs –volume-name=deduped –volume-capacity=5000MB
# ./mount.sdfs -m /srv -v deduped &
# cp -a jdk1.7.0 /root/
# du -sh /root/jdk1.7.0/
200M /root/jdk1.7.0/
# cp -a jdk1.7.0 /srv/
# du -sh /srv/jdk1.7.0/
4.1M /srv/jdk1.7.0/
Great results…!