Operations and Deployment - Revision history

Kennedc1: /* Deployment */

2008-02-01T16:04:58Z

Deployment

← Older revision		Revision as of 11:04, 1 February 2008
Line 41:		Line 41:
	** Tier 2: Sites that manage their own REDDnet depots		** Tier 2: Sites that manage their own REDDnet depots
	** Tier 3: Sites that use their own storage resources as depots		** Tier 3: Sites that use their own storage resources as depots
	** Develop MOU for each tier		** Tier ?: Sites that supply rack space, and basic infrastructure, but management is by ACCRE
			** Develop MOU for each tier - we need to be able to supply reliable storage.

			* Needs
			** What resources
			** What policies
			** What management

	* Investigate new management tools over the next two months		* Investigate new management tools over the next two months

Rgb at 14:45, 31 January 2008

2008-01-31T14:45:18Z

Sheldonp at 16:04, 28 January 2008

2008-01-28T16:04:02Z

← Older revision		Revision as of 11:04, 28 January 2008
Line 33:		Line 33:

	* Gain experience with existing deployment		* Gain experience with existing deployment
			** See "Validation Framework" below.
			** make frequent reports in weekly REDDnet meetings
			** review deployment experience end of March

	* Proposed multi-tiered system for sites for discussion		* Proposed multi-tiered system for sites for discussion
Line 40:		Line 43:
	** Develop MOU for each tier		** Develop MOU for each tier

	* Investigate new ~~monitoring and~~ management tools		* Investigate new management tools over the next two months
	** rsync or similar (short term)		** rsync or similar (short term)
	** Perceus (long term)		** Perceus (long term)
Line 47:		Line 50:
	* Use StorCore, Nagios, iperf, and visualization tools from SC07		* Use StorCore, Nagios, iperf, and visualization tools from SC07
	** Have a statistic page that gathers information from tests and presents them cleanly		** Have a statistic page that gathers information from tests and presents them cleanly
	** ~~Define~~ support for REDDnet		*** Long term project - 6 months to a year?

			* What is required to provide adequate support for REDDnet
			** Want feedback on this.
			** Setting expectations?
			** Actively talk to users over the next couple of months - 6 months - ongoing
			** Develop an initial plan by mid-Feb, then review every 3 months and tweak

	* Create a REDDnet status site, using google maps		* Create a REDDnet status site, using google maps
			** short term just get green dots on a map (what is the priority for this?)
			** longer term - expected to evolve, integrate with the vis site

	* Create an RT site to resolve users' issues		* Create an RT site to resolve users' issues
			** needs to happen quickly. mid-Feb.

	=== Validation Framework ===		=== Validation Framework ===
	* Stress and WAN testing on Production REDDnet		* Stress and WAN testing on Production REDDnet
	** Automated testing with Clyde		** Automated testing with Clyde
	** Real world use		*** excercise system prior to heavy real world use
			*** unfortunately longer term - first of April?
			*** this testing will move to test deployment eventually
			** Real world use (happening now, although not heavy)

	* QA testing on Test REDDnet required before moving into production REDDnet		* QA testing on Test REDDnet required before moving into production REDDnet
	** A stringent set of tests to test both the hardware, OS, IBP, and LStore as throughly as possible (primarily Clyde)		** A stringent set of tests to test both the hardware, OS, IBP, and LStore as throughly as possible (primarily Clyde)
	** Allow users to test using this system		** Allow users to test using this system

Sheldonp: /* Deployment */

2008-01-28T15:40:46Z

Deployment

← Older revision		Revision as of 10:40, 28 January 2008
Line 1:		Line 1:
	=== Deployment ===		=== Deployment ===
	* Bring current deployment up-to-date		* Bring current deployment up-to-date (estimate complete by mid-Feb)
	** Build new image for 2GB internal USB memory		** Build new image for 2GB internal USB memory
	** Design and implement a depot recovery process		*** may be done by All-Hands
			** Design and implement a depot recovery process
			*** This process will be vetted on current deployed hardware
			*** Initial process may be done by All Hands or soon thereafter
			*** Begin with SFASU for initial vetting
	** Send recovery keys out to sites and update the depots		** Send recovery keys out to sites and update the depots
			*** early February
			*** need to recruit one person at each site to assist
	** Set Nagios back up		** Set Nagios back up
	** ~~Develop MOU for current deployment~~		*** need a more stable system before this makes sense
			*** turned off during this transition period to avoid flood of diagnostics
			*** needs a day or two of time, could be done now depending on priority

	* Prepare existing hardware for deployment		* Prepare additional existing hardware for deployment (19 nodes)
	** Update image on internal USB (will use for testing of the above recovery process)		** Update image on internal USB (will use for testing of the above recovery process)
	** Send 6 depots to SFASU with additional PDU		** Send 6 depots to SFASU with additional PDU
	** Find new collaborators/sites		** Find new collaborators/sites for remaining 13.
			*** Ideas?

			* Develop MOU for current deployment (timescale uncertain?)
			** Longer term project

	* Define a standard set of software tools for depots		* Define a standard set of software tools for depots
	** Iperf		** The following also exist
	** Nagios		*** Iperf
	** mtr		*** Nagios
	** other tools~~...~~		*** mtr
			** other tools to be added?
			*** investigating new tools now

	* Gain experience with existing deployment		* Gain experience with existing deployment

	* ~~Discuss a~~ multi-tiered system for sites		* Proposed multi-tiered system for sites for discussion
	** Tier 1: Sites that run their own LServer and Chord ring		** Tier 1: Sites that run their own LServer and Chord ring
	** Tier 2: Sites that manage their own REDDnet depots		** Tier 2: Sites that manage their own REDDnet depots

Rgb at 14:29, 28 January 2008

2008-01-28T14:29:54Z

← Older revision		Revision as of 09:29, 28 January 2008
Line 1:		Line 1:
	= Deployment =		=== Deployment ===
	* Bring current deployment up-to-date		* Bring current deployment up-to-date
	** Build new image for 2GB internal USB memory		** Build new image for 2GB internal USB memory
Line 30:		Line 30:
	** Perceus (long term)		** Perceus (long term)

	= Monitoring =		=== Monitoring ===
	* Use StorCore, Nagios, iperf, and visualization tools from SC07		* Use StorCore, Nagios, iperf, and visualization tools from SC07
	** Have a statistic page that gathers information from tests and presents them cleanly		** Have a statistic page that gathers information from tests and presents them cleanly
Line 39:		Line 39:
	* Create an RT site to resolve users' issues		* Create an RT site to resolve users' issues

	= Validation Framework =		=== Validation Framework ===
	* Stress and WAN testing on Production REDDnet		* Stress and WAN testing on Production REDDnet
	** Automated testing with Clyde		** Automated testing with Clyde

Rgb: /* Validation Framework */

2008-01-28T14:28:21Z

Validation Framework

← Older revision		Revision as of 09:28, 28 January 2008
Line 45:		Line 45:

	* QA testing on Test REDDnet required before moving into production REDDnet		* QA testing on Test REDDnet required before moving into production REDDnet
	** A stringent set of tests to test ~~deployment~~ as throughly as possible (primarily Clyde)		** A stringent set of tests to test both the hardware, OS, IBP, and LStore as throughly as possible (primarily Clyde)
	** Allow users to test using this system		** Allow users to test using this system

Rgb: /* Validation Framework */

2008-01-28T14:27:25Z

Validation Framework

← Older revision		Revision as of 09:27, 28 January 2008
Line 41:		Line 41:
	= Validation Framework =		= Validation Framework =
	* Stress and WAN testing on Production REDDnet		* Stress and WAN testing on Production REDDnet
			** Automated testing with Clyde
			** Real world use

	* QA testing on Test REDDnet required before moving into production REDDnet		* QA testing on Test REDDnet required before moving into production REDDnet
			** A stringent set of tests to test deployment as throughly as possible (primarily Clyde)
			** Allow users to test using this system

Rgb: /* Monitoring */

2008-01-28T14:24:12Z

Monitoring

← Older revision		Revision as of 09:24, 28 January 2008
Line 32:		Line 32:
	= Monitoring =		= Monitoring =
	* Use StorCore, Nagios, iperf, and visualization tools from SC07		* Use StorCore, Nagios, iperf, and visualization tools from SC07
			** Have a statistic page that gathers information from tests and presents them cleanly
			** Define support for REDDnet

	* Create a REDDnet status site, using google maps		* Create a REDDnet status site, using google maps

	* Create an RT site to resolve users' issues		* Create an RT site to resolve users' issues

Rgb: /* Deployment */

2008-01-28T14:16:54Z

Deployment

← Older revision		Revision as of 09:16, 28 January 2008
Line 1:		Line 1:
	= Deployment =		= Deployment =
	* Bring current deployment up-to-date		* Bring current deployment up-to-date
			** Build new image for 2GB internal USB memory
			** Design and implement a depot recovery process
			** Send recovery keys out to sites and update the depots
			** Set Nagios back up
			** Develop MOU for current deployment

	* Prepare existing hardware for deployment		* Prepare existing hardware for deployment
	* ~~Develop MOU~~ for ~~current deployment~~		** Update image on internal USB (will use for testing of the above recovery process)
	* ~~Design and implement a depot recovery process~~		** Send 6 depots to SFASU with additional PDU
			** Find new collaborators/sites

	* Define a standard set of software tools for depots		* Define a standard set of software tools for depots
			** Iperf
			** Nagios
			** mtr
			** other tools...

	* Gain experience with existing deployment		* Gain experience with existing deployment

	* Discuss a multi-tiered system for sites		* Discuss a multi-tiered system for sites
	* ~~Find~~ new ~~collaborators/sites~~		** Tier 1: Sites that run their own LServer and Chord ring
	* ~~Investigate~~ Perceus ~~as an update tool~~		** Tier 2: Sites that manage their own REDDnet depots
			** Tier 3: Sites that use their own storage resources as depots
			** Develop MOU for each tier

			* Investigate new monitoring and management tools
			** rsync or similar (short term)
			** Perceus (long term)

	= Monitoring =		= Monitoring =

Rgb: /* Deployment */

2008-01-25T14:03:55Z

Deployment

← Older revision		Revision as of 09:03, 25 January 2008
Line 7:		Line 7:
	* Gain experience with existing deployment		* Gain experience with existing deployment
	* Discuss a multi-tiered system for sites		* Discuss a multi-tiered system for sites
	* Find new sites		* Find new collaborators/sites
	* Investigate Perceus as an update tool		* Investigate Perceus as an update tool

← Older revision		Revision as of 09:45, 31 January 2008
Line 1:		Line 1:
	=== Deployment ===		=== Deployment ===
	* Bring current deployment up-to-date (estimate complete by mid-Feb)		* '''Bring current deployment up-to-date (estimate complete by mid-Feb)'''
	** Build new image for 2GB internal USB memory		** Build new image for 2GB internal USB memory
	*** may be done by All-Hands		*** may be done by All-Hands
Line 15:		Line 15:
	*** needs a day or two of time, could be done now depending on priority		*** needs a day or two of time, could be done now depending on priority

	* Prepare additional existing hardware for deployment (19 nodes)		* '''Prepare additional existing hardware for deployment (19 nodes)'''
	** Update image on internal USB (will use for testing of the above recovery process)		** Update image on internal USB (will use for testing of the above recovery process)
	** Send 6 depots to SFASU with additional PDU		** Send 6 depots to SFASU with additional PDU
Line 37:		Line 37:
	** review deployment experience end of March		** review deployment experience end of March

	* Proposed multi-tiered system for sites for discussion		* '''Proposed multi-tiered system for sites for discussion'''
	** Tier 1: Sites that run their own LServer and Chord ring		** Tier 1: Sites that run their own LServer and Chord ring
	** Tier 2: Sites that manage their own REDDnet depots		** Tier 2: Sites that manage their own REDDnet depots
Line 48:		Line 48:

	=== Monitoring ===		=== Monitoring ===
	* Use StorCore, Nagios, iperf, and visualization tools from SC07		* '''Use StorCore, Nagios, iperf, and visualization tools from SC07'''
	** Have a statistic page that gathers information from tests and presents them cleanly		** Have a statistic page that gathers information from tests and presents them cleanly
	*** Long term project - 6 months to a year?		*** Long term project - 6 months to a year?

	* What is required to provide adequate support for REDDnet		* '''What is required to provide adequate support for REDDnet'''
	** Want feedback on this.		** Want feedback on this.
	** Setting expectations?		** Setting expectations?
Line 62:		Line 62:
	** longer term - expected to evolve, integrate with the vis site		** longer term - expected to evolve, integrate with the vis site

	* Create an RT site to resolve users' issues		* '''Create an RT site to resolve users' issues'''
	** needs to happen quickly. mid-Feb.		** needs to happen quickly. mid-Feb.

	=== Validation Framework ===		=== Validation Framework ===
	* Stress and WAN testing on Production REDDnet		* '''Stress and WAN testing on Production REDDnet'''
	** Automated testing with Clyde		** Automated testing with Clyde
	*** excercise system prior to heavy real world use		*** excercise system prior to heavy real world use