BWC results
Testing Methodology
Bandwidth tests between REDDnet depots are performed using the ibp_perf tool to emulate depot-to-depot copies. The parameters used are as follows:
- Transfer sizes (MB): 16, 64, 256, 1024, 2048, 4096 (when possible)
- A 32M allocation size is used for all transfers except the 16M (1M) and 64M (2M) cases.
- Number of files transfered: transfer_size/allocation_size
- Concurrent transfer threads: 1, 2, 4, 8, 16, 32
- Block sizes: 1024K, 2048K (1024K for results unless noted)
Each ibp_perf test case is run 5 times in sequence and the results are then averaged before graphing. When simulating depot-to-depot copies using all depots at a particular site, a one-to-one mapping is used in transferring data simultaneously between depots. Thus siteA-depot1 contacts siteB-depot1, siteA-depot2 contacts siteB-depot2, etc.
Full CSV results are available here.
CERN - Vanderbilt (May 15 2009)
Direct tests from CERN to Vandy, aggregated over 4 depots.
Phoebus tests from CERN to Vandy, aggregated over 4 depots. One GW used at ATLA.
Phoebus tests from CERN to Vandy, aggregated over 4 depots. Two GWs used, CHIC-ATLA.
Vanderbilt (Blackhole) - UMich Direct
Direct tests from Vandy to UMich, aggregated throughput over 8 depots.
Vanderbilt (Blackhole) - Caltech
Direct tests from Vandy to Caltech, aggregated throughput over 8 depots.
Phoebus tests from Vandy to Caltech (ALTA-LOSA), aggregated over 8 depots.
CERN - UMich
Direct tests from CERN to UMich, aggregated throughput over 4 depots.
Phoebus tests from CERN to UMich, aggregated throughput over 4 depots.
CERN - Vanderbilt
Direct tests from CERN to Vanderbilt, aggregated throughput over 4 depots. ibp_perf continues to hang on many of the longer transfers and the 2048MB points are omitted where one or more depots failed to complete the test.
The Amsterdam Phoebus gateway is still having hardware issues and results with Phoebus between these sites will be made available when possible. Some individual tests were performed before the Amsterdam host went down:
direct : 1.2Gbps LOSA : 700Mbps CHIC-LOSA : 1.3Gbps AMS-CHIC-LOSA : 2.0Gbps
Performance to the CHIC or LOSA GWs directly is not stellar, but utilizing the Amsterdam GW was looking promising. Once/If the AMS GW comes online again we should expect this to be the ideal testing case for Phoebus.
Vanderbilt - Caltech
Direct tests from Vanderbilt to Caltech, aggregated throughput over 4 depots. Again, ibp_perf continues to hang on many of the longer tests and 4096MBpoints are omitted where one or more depots failed to complete the test. In this case, only vudepot1<->caltech-depot1 had problems with the 4096MB transfers.
Phoebus tests from Vanderbilt to Caltech through ATLA and LOSA GWs, aggregated throughput over 4 depots. I2 path is ATLA-HOUS-LOSA. Here we see Phoebus not performing as well as the direct tests. Seeing a steady improvement as the number of threads increases indicates that perhaps the path from the vandy/caltech sites to either ATLA/LOSA GWs is not performing as expected.
UMich - Caltech
- NOTE: need to run 4096MB tests
Direct tests between UMich and Caltech, aggregated throughput over 8 depots.
Phoebus tests between UMich and Caltech through CHIC and LOSA GWs, aggregated through over 8 depots. I2 path is CHIC-KANS-HOUS-LOSA. Only 4, 8, and 16 stream tests are shown. Due to current underpowered hardware, the Phoebus GWs become bottlenecks and are limited to a little over 3Gbps.