GlusterFS under OVZ without cap_sys_admin capabilities
17. 7. 2015 14:34 | Linux | Komentáře nejsou povolené u textu s názvem GlusterFS under OVZ without cap_sys_admin capabilitiesBy default glusterfs uses xattrs in trusted namespace which is not avilable under OVZ container without cap_sys_admin capabilities, here is quick&dirty howto to build own version of glusterfs under debian which uses user namespace:
# First install some basic dev tools and build depencies:
apt-get install dpkg-dev fuse
apt-get build-dep glusterfs
# get the source of glusterfs packages
apt-get source glusterfs
# I have problems with some GPG keys you can add missing keys by this command:
gpg --keyserver keyserver.ubuntu.com --recv-keys 90CBD8E4
# lets change the sources by dirty find&replace
cd glusterfs-3.5.2/
find . -regex '.*/[^/]*\.[ch]' -exec sed -i 's/"trusted\./"user./g' {} \;
# build the packages
dpkg-source --commit
dpkg-buildpackage
cd ..
# install it
dpkg -i glusterfs-*.deb
working volume:
gluster> volume info Volume Name: shared Type: Replicate Volume ID: 7253c6d6-aec0-4804-9bb5-c4b0150d69bc Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 172.16.0.62:/gluster Brick2: 172.16.0.1:/gluster
some performance testing on vpsFree.cz with ZFS FS plain disk:
iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F /mnt/tmp/f{0,1,2,3,4,5,6,7,8}.ioz
Iozone: Performance Test of File I/O
Version $Revision: 3.430 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Fri Jul 17 11:48:28 2015
Setting no_unlink
Include close in write timing
Include fsync in write timing
No retest option selected
Record Size 64 kB
File size set to 1048576 kB
Command line used: ./iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F /mnt/tmp/f0.ioz /mnt/tmp/f1.ioz /mnt/tmp/f2.ioz /mnt/tmp/f3.ioz /mnt/tmp/f4.ioz /mnt/tmp/f5.ioz /mnt/tmp/f6.ioz /mnt/tmp/f7.ioz /mnt/tmp/f8.ioz
Output is in kBytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 kBytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 8 processes
Each process writes a 1048576 kByte file in 64 kByte records
Children see throughput for 8 initial writers = 53489.47 kB/sec
Parent sees throughput for 8 initial writers = 53204.68 kB/sec
Min throughput per process = 6546.01 kB/sec
Max throughput per process = 6947.46 kB/sec
Avg throughput per process = 6686.18 kB/sec
Min xfer = 1009216.00 kB
Child[0] xfer count = 1024000.00 kB, Throughput = 6642.03 kB/sec
Child[1] xfer count = 1045696.00 kB, Throughput = 6782.18 kB/sec
Child[2] xfer count = 1034880.00 kB, Throughput = 6712.34 kB/sec
Child[3] xfer count = 1022080.00 kB, Throughput = 6629.23 kB/sec
Child[4] xfer count = 1011328.00 kB, Throughput = 6559.43 kB/sec
Child[5] xfer count = 1009216.00 kB, Throughput = 6546.01 kB/sec
Child[6] xfer count = 1048576.00 kB, Throughput = 6947.46 kB/sec
Child[7] xfer count = 1028352.00 kB, Throughput = 6670.79 kB/sec
iozone test complete.
glusterfs volume:
iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F /mnt/shared/f{0,1,2,3,4,5,6,7,8}.ioz
Iozone: Performance Test of File I/O
Version $Revision: 3.430 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Fri Jul 17 12:19:17 2015
Setting no_unlink
Include close in write timing
Include fsync in write timing
No retest option selected
Record Size 64 kB
File size set to 1048576 kB
Command line used: ./iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F /mnt/shared/f0.ioz /mnt/shared/f1.ioz /mnt/shared/f2.ioz /mnt/shared/f3.ioz /mnt/shared/f4.ioz /mnt/shared/f5.ioz /mnt/shared/f6.ioz /mnt/shared/f7.ioz /mnt/shared/f8.ioz
Output is in kBytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 kBytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 8 processes
Each process writes a 1048576 kByte file in 64 kByte records
Children see throughput for 8 initial writers = 23963.82 kB/sec
Parent sees throughput for 8 initial writers = 14037.13 kB/sec
Min throughput per process = 1783.32 kB/sec
Max throughput per process = 5134.14 kB/sec
Avg throughput per process = 2995.48 kB/sec
Min xfer = 364224.00 kB
Child[0] xfer count = 528192.00 kB, Throughput = 2584.27 kB/sec
Child[1] xfer count = 525248.00 kB, Throughput = 2571.77 kB/sec
Child[2] xfer count = 364224.00 kB, Throughput = 1783.32 kB/sec
Child[3] xfer count = 1048576.00 kB, Throughput = 5134.14 kB/sec
Child[4] xfer count = 526400.00 kB, Throughput = 2577.41 kB/sec
Child[5] xfer count = 1003200.00 kB, Throughput = 4907.09 kB/sec
Child[6] xfer count = 526144.00 kB, Throughput = 2576.12 kB/sec
Child[7] xfer count = 373696.00 kB, Throughput = 1829.70 kB/sec
iozone test complete.
test on zfs with xattr=sa:
iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F /mnt/test/f{0,1,2,3,4,5,6,7,8}.ioz
Iozone: Performance Test of File I/O
Version $Revision: 3.430 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Wed Jul 22 08:09:08 2015
Setting no_unlink
Include close in write timing
Include fsync in write timing
No retest option selected
Record Size 64 kB
File size set to 1048576 kB
Command line used: ./iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F /mnt/test/f0.ioz /mnt/test/f1.ioz /mnt/test/f2.ioz /mnt/test/f3.ioz /mnt/test/f4.ioz /mnt/test/f5.ioz /mnt/test/f6.ioz /mnt/test/f7.ioz /mnt/test/f8.ioz
Output is in kBytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 kBytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 8 processes
Each process writes a 1048576 kByte file in 64 kByte records
Children see throughput for 8 initial writers = 26844.98 kB/sec
Parent sees throughput for 8 initial writers = 26766.06 kB/sec
Min throughput per process = 3354.94 kB/sec
Max throughput per process = 3356.57 kB/sec
Avg throughput per process = 3355.62 kB/sec
Min xfer = 1048064.00 kB
Child[0] xfer count = 1048256.00 kB, Throughput = 3355.46 kB/sec
Child[1] xfer count = 1048256.00 kB, Throughput = 3355.55 kB/sec
Child[2] xfer count = 1048384.00 kB, Throughput = 3355.87 kB/sec
Child[3] xfer count = 1048064.00 kB, Throughput = 3354.94 kB/sec
Child[4] xfer count = 1048576.00 kB, Throughput = 3356.57 kB/sec
Child[5] xfer count = 1048256.00 kB, Throughput = 3355.47 kB/sec
Child[6] xfer count = 1048256.00 kB, Throughput = 3355.65 kB/sec
Child[7] xfer count = 1048256.00 kB, Throughput = 3355.47 kB/sec
iozone test complete.
No Comments yet
Sorry, the comment form is closed at this time.
Powered by WordPress with Pool theme design by Borja Fernandez.
Entries and comments feeds.
Valid XHTML and CSS. ^Top^