BWC results

From ReddNet
Jump to navigation Jump to search

Testing Methodology


Bandwidth tests between REDDnet depots are performed using the ibp_perf tool to emulate depot-to-depot copies. The parameters used are as follows:

  • Transfer sizes (MB): 16, 64, 256, 1024, 2048, 4096 (when possible)
    • A 32M allocation size is used for all transfers except the 16M (1M) and 64M (2M) cases.
  • Number of files transfered: transfer_size/allocation_size
  • Concurrent transfer threads: 1, 2, 4, 8, 16, 32
  • Block sizes: 1024K, 2048K (1024K for results unless noted)


Each ibp_perf test case is run 5 times in sequence and the results are then averaged before graphing. When simulating depot-to-depot copies using all depots at a particular site, a one-to-one mapping is used in transferring data simultaneously between depots. Thus siteA-depot1 contacts siteB-depot1, siteA-depot2 contacts siteB-depot2, etc.

Full CSV results are available here.

CERN - Vanderbilt (May 15 2009)


Direct tests from CERN to Vandy, aggregated over 4 depots.


Cern bh read NP.jpg PDF Cern bh write NP.jpg PDF


Phoebus tests from CERN to Vandy, aggregated over 4 depots. One GW used at ATLA.


Cern bh read ATLA WP.jpg PDF Cern bh write ATLA WP.jpg PDF


Phoebus tests from CERN to Vandy, aggregated over 4 depots. Two GWs used, CHIC-ATLA.


Cern bh read ATLA CHIC WP.jpg PDF Cern bh write ATLA CHIC WP.jpg PDF

Vanderbilt (Blackhole) - UMich Direct


Direct tests from Vandy to UMich, aggregated throughput over 8 depots.

Bh umich read NP.jpg PDF Bh umich write NP.jpg PDF

Vanderbilt (Blackhole) - Caltech


Direct tests from Vandy to Caltech, aggregated throughput over 8 depots.

Bh calt read NP.jpg PDF Bh calt write NP.jpg PDF

Phoebus tests from Vandy to Caltech (ALTA-LOSA), aggregated over 8 depots.


Bh calt read WP.jpg PDF Bh calt write WP.jpg PDF

CERN - UMich


Direct tests from CERN to UMich, aggregated throughput over 4 depots.


Cern umich read NP.jpg PDF Cern umich write NP.jpg PDF


Phoebus tests from CERN to UMich, aggregated throughput over 4 depots.


Cern umich read WP.jpg PDF Cern umich write WP.jpg PDF

CERN - Vanderbilt


Direct tests from CERN to Vanderbilt, aggregated throughput over 4 depots. ibp_perf continues to hang on many of the longer transfers and the 2048MB points are omitted where one or more depots failed to complete the test.

Cern vandy read NP.jpg PDF Cern vandy write NP.jpg PDF

The Amsterdam Phoebus gateway is still having hardware issues and results with Phoebus between these sites will be made available when possible. Some individual tests were performed before the Amsterdam host went down:

direct        : 1.2Gbps
LOSA          : 700Mbps
CHIC-LOSA     : 1.3Gbps
AMS-CHIC-LOSA : 2.0Gbps 


Performance to the CHIC or LOSA GWs directly is not stellar, but utilizing the Amsterdam GW was looking promising. Once/If the AMS GW comes online again we should expect this to be the ideal testing case for Phoebus.

Vanderbilt - Caltech


Direct tests from Vanderbilt to Caltech, aggregated throughput over 4 depots. Again, ibp_perf continues to hang on many of the longer tests and 4096MBpoints are omitted where one or more depots failed to complete the test. In this case, only vudepot1<->caltech-depot1 had problems with the 4096MB transfers.

Vandy caltech read NP.jpg PDF Vandy caltech write NP.jpg PDF


Phoebus tests from Vanderbilt to Caltech through ATLA and LOSA GWs, aggregated throughput over 4 depots. I2 path is ATLA-HOUS-LOSA. Here we see Phoebus not performing as well as the direct tests. Seeing a steady improvement as the number of threads increases indicates that perhaps the path from the vandy/caltech sites to either ATLA/LOSA GWs is not performing as expected.

Vandy caltech read WP.jpg PDF Vandy caltech write WP.jpg PDF


UMich - Caltech


  • NOTE: need to run 4096MB tests

Direct tests between UMich and Caltech, aggregated throughput over 8 depots.


Umich calt 8 read.jpg PDF

Umich calt 8 write.jpg PDF


Phoebus tests between UMich and Caltech through CHIC and LOSA GWs, aggregated through over 8 depots. I2 path is CHIC-KANS-HOUS-LOSA. Only 4, 8, and 16 stream tests are shown. Due to current underpowered hardware, the Phoebus GWs become bottlenecks and are limited to a little over 3Gbps.


Umich calt read WP.jpg PDF

Umich calt write WP.jpg PDF