L-Store Upload Structured Testing

2007-07-19T15:16:47Z

Schoenj3: get rid of annoying list of links

= Objective =

This is the first set of tests for Benchmarking L-Store system performance. We are setting up Upload testing to begin with.

= Parameters =

1. File Size:

* Small: 1KB file will be used
* Medium: 0.5MB file will be used
* Large: 260MB file will be used

2. Number of Files

* Small: Essentially we will use the number of uploads set up as a surrogate for number of files. 50
* Medium: 200
* Large: 1000

3. Number of threads

* Small: 1
* Medium: 20
* Large: 60

*Experimental Design: Full Factorial (3*3*3=27 test cases). Each test case would be run 30 times to collect statistically significant data.

Please watch this page as we post results from our runs.

= Results for small uploads: 10 Threads =

* Number of Threads: 10
* Number of Files : 1500 (30 samples of 50 uploads)
* Size of file: 1 KB
* Current Status: Done
* Time of Completion: 14.00 CST, June 20th 2006

{| class="wikitable" style="text-align:center"
|-
!bgcolor=red | Type of Test || bgcolor=blue | Number of files || bgcolor=grey | Average File Size (MB) || bgcolor=grey | Average Latency (sec) || bgcolor=grey | Median Latency (sec) || bgcolor=grey | Maximum Latency (sec) || bgcolor=grey | Std Deviation on Latency
|-
!bgcolor=red | small_upload|| bgcolor=blue | 1500|| bgcolor=grey | 0.001|| bgcolor=grey | 2.8|| bgcolor=grey | 2.7|| bgcolor=grey | 11.0|| bgcolor=grey | 0.63
|-
|}

= Results for medium uploads: 10 Threads =

* Number of Threads: 10
* Number of Files : 1500 (30 samples of 50 uploads)
* Size of file: 500 KB
* Current Status: Done
* Time of Completion: 17.00 CST, June 20th 2006

{| class="wikitable" style="text-align:center"
|-
!bgcolor=red | Type of Test || bgcolor=blue | Number of files || bgcolor=grey | Average File Size (MB) || bgcolor=grey | Average Latency (sec) || bgcolor=grey | Median Latency (sec) || bgcolor=grey | Maximum Latency (sec) || bgcolor=grey | Std Deviation on Latency
|-
!bgcolor=red | med_upload|| bgcolor=blue | 1500|| bgcolor=grey | 0.5|| bgcolor=grey | 3.0|| bgcolor=grey | 3.0|| bgcolor=grey | 10.0|| bgcolor=grey | 0.45
|-
|}

= Results for large uploads: 10 Threads =

* Number of Threads: 10
* Number of Files : 1500 (30 samples of 50 uploads)
* Size of file: 250 MB
* Current Status: In Progress
* Expected : June 21st 2006
* Time of Completion: 9.00am CST, 21st June 2006

{| class="wikitable" style="text-align:center"
|-
!bgcolor=red | Type of Test || bgcolor=blue | Number of files || bgcolor=grey | Average File Size (MB) || bgcolor=grey | Average Latency (sec) || bgcolor=grey | Median Latency (sec) || bgcolor=grey | Maximum Latency (sec) || bgcolor=grey | Std Deviation on Latency
|-
!bgcolor=red | large_upload|| bgcolor=blue | 1500|| bgcolor=grey | 250.0|| bgcolor=grey | 27.0|| bgcolor=grey | 27.0|| bgcolor=grey | 47.0|| bgcolor=grey | 1.2
|-
|}

REDDnet at Work Page

2007-07-19T15:08:40Z

Schoenj3:

== Organization ==

* [[REDDnet Personnel]]

=== Steering Committee ===

* Members: Martin Swany (Delaware), Rick Cavanaugh (Florida), Shawn McKee (Michigan), John Blondin (NCState), Hunter Hagewood (Nevoa), P. R. Blackwell (Stephen F. Austin), Micah Beck, Terry Moore (Tennessee), Paul Sheldon [Chair], Pheobe Stewart, Alan Tackett (VU)
* Meeting notes are available on the [[REDDnet Meetings and Minutes Page]]

=== Software Development and Support Working Group ===

* WG Members: Alan Tackett (Level 2 Manager), Micah Beck, Chip Cox, Larry Dawson, Santiago de Ledesma, Hunter Hagewood, Surya Pathak
* [[REDDnet Software Development and Support Working Group|Activities]] -- workplan, results,...
* [[REDDnet Software Release Schedule|Release Schedule]] -- anticipated schedule of releases and functionality of each release

=== Deployment and Monitoring Working Group ===

* WG Members: Bobby Brown (Level 2 Manager), Surya Pathak, Martin Swany
* [[REDDnet Deployment and Monitoring Working Group|Activities]] -- workplan, results,...
* [[REDDnet Hardware Release Schedule|Release Schedule]] -- anticipated schedule of system "releases"
* [[TerryDeployIdeas|Terry e-mail with suggestions on deployment]]

=== Application Integration Working Group ===

* WG Members: Terry Moore (Level 2 Manager), P. R. Blackwell, Rick Cavanaugh, Dan Engh, Shawn McKee, Paul Sheldon
* [[REDDnet Application Integration Working Group|Activities]] -- workplan, results,...
* [[Action Plans from December REDDnet Meeting]]

== Meetings and Minutes ==

* [[REDDnet Meetings and Minutes Page]]

== Events ==
=== Collaboration Meetings ===
[[REDDnet Tools and Applications Meeting Spring 2007]] "April 24" 6:30am-9:30pm, Salon D/E
**Crystal Gateway Marriott, Arlington, VA.
**In coordination with the [http://events.internet2.edu/2007/spring-mm/ Spring 2007 Internet2 Member Meeting]
*[[REDDnet Tools and Applications Meeting 2006]] December 4, 8:00am-5:00pm
**Hyatt Regency McCormick Place, Chicago, IL.
**In coordination with the [http://events.internet2.edu/2006/fall-mm/index.html Fall 2006 Internet2 Member Meeting]

=== Past ===
*[[SC06]] Tampa, FL Nov. 11-18, 2006

=== Future (including those under consideration) ===
*[http://www.digitalpreservation.gov/ NDIIPP partners meeting] --- January ?? 2007
**NDIIPP: The National Digital Information Infrastructure and Preservation Program
**Demo and Presentation --- Alan Tackett
*[http://cms.cern.ch/iCMS/ CMS Week] -- February 26 - March 2, 2007 at CERN.
**Dan should present work on L-Store integration into CMS software
**which sessions? need to find out
**need to lay groundwork to get the talk on the agenda
*[https://indico.fnal.gov/conferenceDisplay.py?confId=468 OSG Consortium Meeting] --- March 5-8, 2007 at UC San Diego
** Demo
** Sheldon will be attending, others should as well!
** need to work on integration into OSG before this meeting
*[http://www.americaview.org/ AmericaView Annual Winter Meeting]-- March 5

ReddNet - User contributions [en]

L-Store Upload Structured Testing

REDDnet at Work Page