193
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     1 Variables:
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     2 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     3 Outer:
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     4 - ADDITIONAL_PATH
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     5 - MD5
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     6 - DEFAULT_SITE
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     7 - BASE_DIR
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     8 - MOEFETCHVERSION
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
     9 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    10 Functions:
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    11 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    12 - Msg_Welcome
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    13 	- Welcome message (MOEFETCHVERSION)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    14 - Err_Help
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    15 - Err_Fatal
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    16 - Generate_Link
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    17 	- chdir to ${BASE_DIR}/temp
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    18 	- fetch xml with wget
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    19 	- xsltproc the xml
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    20 - Check_Tools
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    21 	- Check if MD5 is empty - if empty: check os
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    22 		- *BSD: md -r
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    23 		- Linux/SunOS: md5sum
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    24 		- Anything else: Err_Fatal
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    25 	- Get md5 command (MD5_COMMAND)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    26 	- Check availability of needed tools
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    27 		- cut 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    28 		- sed 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    29 		- wc 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    30 		- wget 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    31 		- xsltproc 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    32 		- xargs 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    33 		- rm 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    34 		- mkdir 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    35 		- chown 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    36 		- comm 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    37 		- grep 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    38 		- date 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    39 		- MD5_COMMAND
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    40 	- Check for grep usability 
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    41 	- TODO: greplace grep -f with POSIX compatible
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    42 - Check_Folders
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    43 	- Check BASE_DIR ownership
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    44 	- Check: existance (create if not exist) and ownership (apply globally writable permission) of BASE_DIR/:
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    45 		- temp
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    46 		- trash
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    47 		- deleted
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    48 		- SITE_DIR/TARGET_DIR
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    49 	- Check if "BASE_DIR/SITE_DIR/TARGET_DIR" is empty: if empty, ISNEW=1
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    50 	- Create temporary files: BASE_DIR/temp/${SITE_DIR}-${TARGET_DIR}-:
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    51 		- error
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    52 		- ok
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    53 		- list
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    54 		- newlist
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    55 - Cleanup_Repository
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    56 	- TRASH_DIR: ${BASE_DIR}/trash/${SITE_DIR}-${TARGET_DIR}-%Y%m%d-%H.%M
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    57 	- create trash folder
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    58 	- check if file in "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}/ fulfills [a-f0-9]{32}\..* or is a folder
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    59 	- move to trash if is trash
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    60 	- check if file is contained in ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-list
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    61 	- if is not, move to trash
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    62 - Check_Files
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    63 	- if not ISNEW (empty target folder):
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    64 		- if not NOCLEAN (not skipping cleanup:
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    65 			- Call Clean_Repository
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    66 		- chdir to target folder
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    67 		- TODO: chdir-free operation
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    68 		- empty ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-err
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    69 		- check file in current directory (${BASE_DIR}/${SITE_DIR}/${TARGET_DIR})
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    70 			- skip if not correct file ([a-f0-9]{32}\..*)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    71 			- put into ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-error for every error files
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    72 		- remove the files
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    73 		- chdir to temp folder
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    74 		- list ${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}, compare with error, exclude the errors, put into ${SITE_DIR}-${TARGET_DIR}-ok
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    75 		- get list of new files - compare with ${SITE_DIR}-${TARGET_DIR}-list
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    76 		- TODO: remove ls, grep -f dependencies
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    77 	- if ISQUICK: skip check
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    78 	- if not ISQUICK: print 'empty repository'
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    79 	- copy ${SITE_DIR}-${TARGET_DIR}-list to ${SITE_DIR}-${TARGET_DIR}-newlist
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    80 - Fetch_Images
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    81 	- chdir to ${BASE_DIR}/temp
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    82 	- check if ${SITE_DIR}-${TARGET_DIR}-newlist is empty -> stop
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    83 	- chdir to ${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    84 	- start wget: wget -e continue=on -bi "${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-newlist" -o "${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}.log"
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    85 - Init
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    86 	- Add path (PATH)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    87 	- Check command (fetch/check/quickfetch/* - JOB)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    88 	- Check site (-s <site> or default - SITE)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    89 	- Check if not clean folder (-nc/no clean - NOCLEAN)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    90 	- Get tags (TAGS)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    91 	- Check site - if SITE empty then set default (SITE=DEFAULT_SITE)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    92 	- TODO: Validate SITE
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    93 	- Check tag - if TAGS empty then Err_Fatal
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    94 	- Get BASE_DIR: default to PWD - fallback to HOME
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    95 	- Validate BASE_DIR: must absolute path
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    96 	- Get TARGET_DIR: escape TAGS (replace / with _)
 | 
edhoprima@gmail.com <edhoprima@gmail.com> 
parents:  
diff
changeset
 
 | 
    97 	- Get SITE_DIR: escape SITE (remove ending /, replace / with _)
 |