view Script_Explanation @ 193:ac6533a8fb51

- Documentation - Cleanup
author edhoprima@gmail.com <edhoprima@gmail.com>
date Mon, 29 Jun 2009 10:49:40 +0000
parents
children
line wrap: on
line source

Variables:

Outer:
- ADDITIONAL_PATH
- MD5
- DEFAULT_SITE
- BASE_DIR
- MOEFETCHVERSION

Functions:

- Msg_Welcome
	- Welcome message (MOEFETCHVERSION)
- Err_Help
- Err_Fatal
- Generate_Link
	- chdir to ${BASE_DIR}/temp
	- fetch xml with wget
	- xsltproc the xml
- Check_Tools
	- Check if MD5 is empty - if empty: check os
		- *BSD: md -r
		- Linux/SunOS: md5sum
		- Anything else: Err_Fatal
	- Get md5 command (MD5_COMMAND)
	- Check availability of needed tools
		- cut 
		- sed 
		- wc 
		- wget 
		- xsltproc 
		- xargs 
		- rm 
		- mkdir 
		- chown 
		- comm 
		- grep 
		- date 
		- MD5_COMMAND
	- Check for grep usability 
	- TODO: greplace grep -f with POSIX compatible
- Check_Folders
	- Check BASE_DIR ownership
	- Check: existance (create if not exist) and ownership (apply globally writable permission) of BASE_DIR/:
		- temp
		- trash
		- deleted
		- SITE_DIR/TARGET_DIR
	- Check if "BASE_DIR/SITE_DIR/TARGET_DIR" is empty: if empty, ISNEW=1
	- Create temporary files: BASE_DIR/temp/${SITE_DIR}-${TARGET_DIR}-:
		- error
		- ok
		- list
		- newlist
- Cleanup_Repository
	- TRASH_DIR: ${BASE_DIR}/trash/${SITE_DIR}-${TARGET_DIR}-%Y%m%d-%H.%M
	- create trash folder
	- check if file in "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}/ fulfills [a-f0-9]{32}\..* or is a folder
	- move to trash if is trash
	- check if file is contained in ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-list
	- if is not, move to trash
- Check_Files
	- if not ISNEW (empty target folder):
		- if not NOCLEAN (not skipping cleanup:
			- Call Clean_Repository
		- chdir to target folder
		- TODO: chdir-free operation
		- empty ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-err
		- check file in current directory (${BASE_DIR}/${SITE_DIR}/${TARGET_DIR})
			- skip if not correct file ([a-f0-9]{32}\..*)
			- put into ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-error for every error files
		- remove the files
		- chdir to temp folder
		- list ${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}, compare with error, exclude the errors, put into ${SITE_DIR}-${TARGET_DIR}-ok
		- get list of new files - compare with ${SITE_DIR}-${TARGET_DIR}-list
		- TODO: remove ls, grep -f dependencies
	- if ISQUICK: skip check
	- if not ISQUICK: print 'empty repository'
	- copy ${SITE_DIR}-${TARGET_DIR}-list to ${SITE_DIR}-${TARGET_DIR}-newlist
- Fetch_Images
	- chdir to ${BASE_DIR}/temp
	- check if ${SITE_DIR}-${TARGET_DIR}-newlist is empty -> stop
	- chdir to ${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}
	- start wget: wget -e continue=on -bi "${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-newlist" -o "${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}.log"
- Init
	- Add path (PATH)
	- Check command (fetch/check/quickfetch/* - JOB)
	- Check site (-s <site> or default - SITE)
	- Check if not clean folder (-nc/no clean - NOCLEAN)
	- Get tags (TAGS)
	- Check site - if SITE empty then set default (SITE=DEFAULT_SITE)
	- TODO: Validate SITE
	- Check tag - if TAGS empty then Err_Fatal
	- Get BASE_DIR: default to PWD - fallback to HOME
	- Validate BASE_DIR: must absolute path
	- Get TARGET_DIR: escape TAGS (replace / with _)
	- Get SITE_DIR: escape SITE (remove ending /, replace / with _)