EntityFS 1.1.3 test report

2009.07.15
Karl Gustafsson

Unit tests
  64 bit Linux
  32 bit Linux
  64 bit Solaris 10 (x64)
  64 bit Mac OS X (x64)
  64 bit Windows Vista
  32 bit Windows Vista

Unit tests

Unit tests were run as described in the Programmer's guide.

64 bit Linux

OS:

Xubuntu 9.04 64bit, Linux 2.6.28-11-generic SMP

File system:

tmpfs

Hardware:

Intel Core 2 Quad Q9550 @ 2.83GHz
8192 Mb RAM
Western Digital Raptor (3.5", 10,000 RPM, SATA)

JVM	Results
Sun JDK 1.5.0_19 (64 bit)	OK (14506/14506)
Sun JDK 1.6.0_14 (64 bit)	OK (14507/14507)
IBM J9 JVM build 2.3 1.5.0 SR9-SSU (64 bit)	failed* (268/14506)
IBM J9 JVM build 2.4 1.6.0 SR5 (64 bit)	OK (14507/14507)
OpenJDK 6b14-1.4.1-0ubuntu10 (64 bit)	OK, (14507/14507)

* All test threads were dead after 268 test suites.

32 bit Linux

OS:

Xubuntu 9.04 32bit, Linux 2.6.28-13-generic

File system:

tmpfs

Hardware:

Intel Pentium M 2.0 GHz stepping 08
2048 Mb RAM
Hitachi HTE726060M9AT00 (2.5", 7,200 RPM, ATA)

JVM	Results
Sun JDK 1.5.0_19	OK (14506/14506)
Sun JDK 1.6.0_14	OK (14507/14507)
IBM J9 JVM build 2.3 1.5.0 SR9-SSU	failed* (87/14506)
IBM J9 JVM build 2.4 1.6.0 SR5	OK (14507/14507)
OpenJDK 6b14-1.4.1-0ubuntu10	OK, (14507/14507)

* All test threads were dead after 87 test suites.

64 bit Solaris 10

OS:

Solaris 10 5/09

File system:

UFS

Hardware:

Intel Xeon E5420 @ 2.5GHz
8192 Mb RAM
RAID 10 array

JVM	Results
Sun JDK 1.5.0_19 (64 bit)	OK (14506/14506)
Sun JDK 1.6.0_14 (64 bit)	OK (14507/14507)

64 bit Mac OS X

OS:

Mac OS X 10.5.7

File system:

HFS

Hardware:

Intel Core 2 Duo @ 2GHz
2048 Mb RAM

JVM	Results
Apple JDK 1.5.0_19_b02-304 (64 bit)	OK (14506/14506)
Apple JDK 1.6.0_13_b03-211 (64 bit)	OK (14507/14507)

64 bit Windows Vista

OS:

Windows Vista Home Premium 64bit SP2

File system:

NTFS

Hardware:

Intel Core 2 Duo T5600 @ 1.83GHz stepping 06
2048 Mb RAM
Samsung HM160JI (2.5", 5,400 RPM, SATA)

JVM	Results
Sun JDK 1.5.0_19 (64 bit)	OK (14506/14506)
Sun JDK 1.6.0_14 (64 bit)	OK (14507/14507)

32 bit Windows Vista

OS:

Windows Vista Enterprise SP2

File system:

NTFS

Hardware:

Intel Core2 Duo U7600 @ 1.2Ghz
2048 Mb RAM
Toshiba MK1011GAH (1.8", 4,200 RPM, ATA)

JVM	Results
Sun JDK 1.5.0_19	OK (14506/14506)
Sun JDK 1.6.0_14	OK (14507/14507)

Performance tests

Performance tests were run as described in the Programmer's guide. The tests run are:

1. Create 30000 files in a directory using Directory.newEntity
2. Call getEntity on each of the 30000 files
3. Delete 30000 files using IteratorDeleter
4. Create 30000 directories in a directory using Directory.newEntity
5. Call getEntity on each of the 30000 directories
6. Delete 30000 directories using IteratorDeleter
7. Copy Zip archive (Linux kernel source distribution) into file system
8. Unzip Zip archive into file system by creating a Zip file system on it and using the IteratorCopier for copying entities
9. Create Zip archive of file tree created in the previous test using ZipCreator
10. Copy the entire unzipped tree within the file system using IteratorCopier
11. Delete the unzipped tree using IteratorDeleter
12. Copy the Zip archive within the file system using Entities.copy
13. Create 128 Mb empty file
14. Copy 128 Mb empty file within file system
15. Read 128 Mb empty file in 8 kb chunks
16. Create 32 Mb empty file
17. Copy 32 Mb empty file within file system
18. Read 32 Mb empty file in 8 kb chunks
19. Create 8 Mb empty file
20. Copy 8 Mb empty file within file system
21. Read 8 Mb empty file in 8 kb chunks
22. Split 16 Mb empty file into 1 kb chunks
23. Split 16 Mb empty file into 2 kb chunks
24. Split 16 Mb empty file into 4 kb chunks
25. Split 16 Mb empty file into 8 kb chunks

Each test is run five times and the median time is recorded. The test suite is run for different (read/write) file system configurations and a reference test suite that only uses plain Java methods is also run. The test suite is based on the test suite described in this Linux Gazette article.

The goal of the performance testing have not been to measure the absolute performance of the file system operations. It has just been to verify that EntityFS methods do not perform significantly worse than their Java counterparts.

The tests were run in an unmodified OS in single user mode.

64 bit Linux

OS:

Xubuntu 9.04 64bit, Linux 2.6.28-11-generic SMP

File system:

Ext3

Hardware:

Intel Core 2 Quad Q9550 @ 2.83GHz
8192 Mb RAM
Western Digital Raptor (3.5", 10,000 RPM, SATA)

Reference values (median times in milliseconds):

JVM	1	2	3	4	5	6	7	8	9	10	11	12
Sun JDK 1.5.0_19 (64 bit)	818	-	608	3241	-	1925	184	4058	16487	2188	828	186
Sun JDK 1.6.0_14 (64 bit)	794	-	606	3148	-	1909	185	3984	15789	2124	917	180
JVM	13	14	15	16	17	18	19	20	21	22	23	24	25
Sun JDK 1.5.0_19 (64 bit)	254	255	85	75	76	24	19	19	6	739	383	193	117
Sun JDK 1.6.0_14 (64 bit)	257	250	95	64	71	25	18	19	6	716	374	184	104

FSRWFileSystemBuilder, default settings (times in milliseconds, time/ref time):

JVM	1	2	3	4	5	6	7	8	9	10	11	12
Sun JDK 1.5.0_19 (64 bit)	1369 (1.67)	156	1284 (2.11)	3797 (1.17)	143	3530 (1.83)	173 (0.94)	12126 (2.99)	23209 (1.41)	8289 (3.79)	1587 (1.92)	178 (0.96)
Sun JDK 1.6.0_14 (64 bit)	1322 (1.66)	155	1114 (1.84)	5361 (1.70)	170	3388 (1.77)	175 (0.95)	11795 (2.96)	23017 (1.46)	7794 (3.67)	1517 (1.65)	181 (1.01)
JVM	13	14	15	16	17	18	19	20	21	22	23	24	25
Sun JDK 1.5.0_19 (64 bit)	238 (0.94)	253 (0.99)	91 (1.07)	62 (0.83)	68 (0.89)	53 (1.04)	49 (1.02)	56 (1.04)	24 (1.00)	1203 (1.63)	629 (1.64)	339 (1.76)	168 (1.44)
Sun JDK 1.6.0_14 (64 bit)	241 (0.94)	209 (1.00)	91 (0.96)	60 (0.94)	78 (1.10)	24 (0.96)	19 (1.06)	20 (1.05)	6 (1.00)	1190 (1.66)	596 (1.59)	284 (1.54)	154 (1.48)

FSRWFileSystemBuilder, no entity validity controls (times in milliseconds, time/ref time):

JVM	1	2	3	4	5	6	7	8	9	10	11	12
Sun JDK 1.5.0_19 (64 bit)	1173 (1.43)	43	853 (1.40)	3574 (1.10)	41	2884 (1.50)	171 (0.93)	10897 (2.69)	22808 (1.38)	6643 (3.04)	1169 (1.41)	183 (0.98)
Sun JDK 1.6.0_14 (64 bit)	1095 (1.38)	33	838 (1.38)	5793 (1.84)	35	2712 (1.42)	181 (0.98)	10691 (2.68)	22453 (1.42)	6427 (3.02)	1110 (1.21)	187 (1.04)
JVM	13	14	15	16	17	18	19	20	21	22	23	24	25
Sun JDK 1.5.0_19 (64 bit)	237 (0.93)	251 (0.98)	84 (0.99)	69 (0.92)	69 (0.91)	24 (1.00)	19 (1.00)	20 (1.05)	6 (1.00)	1058 (1.43)	564 (1.47)	281 (1.46)	158 (1.35)
Sun JDK 1.6.0_14 (64 bit)	237 (0.92)	249 (1.00)	87 (0.92)	67 (1.05)	79 (1.11)	24 (0.96)	20 (1.11)	20 (1.05)	6 (1.00)	1020 (1.42)	544 (1.45)	273 (1.48)	146 (1.40)

FSRWFileSystemBuilder, events, locking (times in milliseconds, time/ref time):

JVM	1	2	3	4	5	6	7	8	9	10	11	12
Sun JDK 1.5.0_19 (64 bit)	1685 (2.05)	175	1604 (2.64)	4221 (1.30)	171	3929 (2.04)	197 (1.07)	12609 (3.11)	23635 (1.43)	9534 (4.36)	1901 (2.30)	205 (1.10)
Sun JDK 1.6.0_14 (64 bit)	1799 (2.27)	183	1656 (2.73)	4548 (1.44)	173	4546 (2.38)	206 (1.11)	12204 (3.06)	23282 (1.47)	8865 (4.17)	1876 (2.05)	200 (1.11)
JVM	13	14	15	16	17	18	19	20	21	22	23	24	25
Sun JDK 1.5.0_19 (64 bit)	252 (0.99)	280 (1.10)	86 (1.01)	62 (0.83)	84 (1.11)	23 (0.96)	20 (1.05)	23 (1.21)	6 (1.00)	1431 (1.94)	718 (1.87)	361 (1.87)	187 (1.60)
Sun JDK 1.6.0_14 (64 bit)	236 (0.92)	274 (1.10)	82 (0.86)	66 (1.03)	81 (1.14)	24 (0.96)	19 (1.06)	22 (1.16)	6 (1.00)	1455 (2.03)	716 (1.91)	354 (1.92)	194 (1.87)

Default RamFileSystemBuilder settings (times in milliseconds, time/ref time):

JVM	1	2	3	4	5	6	7	8	9	10	11	12
Sun JDK 1.5.0_19 (64 bit)	92 (0.11)	52	95 (0.15)	96 (0.02)	46	122 (0.06)	163 (0.89)	6856 (1.69)	15829 (0.96)	2109 (0.96)	107 (0.13)	135 (0.73)
Sun JDK 1.6.0_14 (64 bit)	112 (0.14)	49	105 (0.17)	96 (0.03)	47	123 (0.06)	196 (1.06)	6454 (1.62)	15800 (1.00)	1900 (0.89)	105 (0.11)	169 (0.94)
JVM	13	14	15	16	17	18	19	20	21	22	23	24	25
Sun JDK 1.5.0_19 (64 bit)	175 (0.69)	219 (0.86)	109 (1.28)	36 (0.48)	54 (0.71)	28 (1.17)	8 (0.42)	16 (0.84)	11 (1.83)	250 (0.34)	129 (0.34)	73 (0.38)	57 (0.49)
Sun JDK 1.6.0_14 (64 bit)	183 (0.71)	225 (0.90)	87 (0.92)	22 (0.34)	100 (1.40)	62 (2.48)	4 (0.22)	5 (0.26)	5 (0.83)	216 (0.30)	107 (0.29)	70 (0.38)	51 (0.49)

RamFileSystemBuilder, events, locking (times in milliseconds, time/ref time):

JVM	1	2	3	4	5	6	7	8	9	10	11	12
Sun JDK 1.5.0_19 (64 bit)	376 (0.46)	58	393 (0.65)	380 (0.12)	58	521 (0.27)	156 (0.85)	7513 (1.85)	16227 (0.98)	2935 (1.34)	362 (0.44)	151 (0.81)
Sun JDK 1.6.0_14 (64 bit)	409 (0.51)	27	408 (0.67)	403 (0.12)	26	539 (0.28)	121 (0.65)	6863 (1.72)	16090 (1.02)	2555 (1.20)	344 (0.38)	168 (0.93)
13	14	15	16	17	18	19	20	21	22	23	24	25
Sun JDK 1.5.0_19 (64 bit)	184 (0.72)	204 (0.80)	104 (1.22)	40 (0.53)	63 (0.83)	30 (1.25)	10 (0.53)	20 (1.05)	14 (2.33)	415 (0.56)	225 (0.59)	118 (0.61)	86 (0.74)
Sun JDK 1.6.0_14 (64 bit)	168 (0.65)	234 (0.94)	89 (0.94)	30 (0.47)	68 (0.96)	36 (1.44)	3 (0.17)	8 (0.42)	5 (0.83)	397 (0.55)	206 (0.55)	104 (0.57)	63 (0.61)