193
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
1 Variables:
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
2
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
3 Outer:
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
4 - ADDITIONAL_PATH
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
5 - MD5
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
6 - DEFAULT_SITE
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
7 - BASE_DIR
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
8 - MOEFETCHVERSION
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
9
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
10 Functions:
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
11
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
12 - Msg_Welcome
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
13 - Welcome message (MOEFETCHVERSION)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
14 - Err_Help
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
15 - Err_Fatal
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
16 - Generate_Link
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
17 - chdir to ${BASE_DIR}/temp
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
18 - fetch xml with wget
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
19 - xsltproc the xml
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
20 - Check_Tools
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
21 - Check if MD5 is empty - if empty: check os
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
22 - *BSD: md -r
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
23 - Linux/SunOS: md5sum
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
24 - Anything else: Err_Fatal
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
25 - Get md5 command (MD5_COMMAND)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
26 - Check availability of needed tools
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
27 - cut
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
28 - sed
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
29 - wc
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
30 - wget
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
31 - xsltproc
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
32 - xargs
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
33 - rm
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
34 - mkdir
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
35 - chown
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
36 - comm
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
37 - grep
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
38 - date
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
39 - MD5_COMMAND
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
40 - Check for grep usability
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
41 - TODO: greplace grep -f with POSIX compatible
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
42 - Check_Folders
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
43 - Check BASE_DIR ownership
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
44 - Check: existance (create if not exist) and ownership (apply globally writable permission) of BASE_DIR/:
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
45 - temp
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
46 - trash
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
47 - deleted
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
48 - SITE_DIR/TARGET_DIR
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
49 - Check if "BASE_DIR/SITE_DIR/TARGET_DIR" is empty: if empty, ISNEW=1
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
50 - Create temporary files: BASE_DIR/temp/${SITE_DIR}-${TARGET_DIR}-:
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
51 - error
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
52 - ok
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
53 - list
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
54 - newlist
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
55 - Cleanup_Repository
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
56 - TRASH_DIR: ${BASE_DIR}/trash/${SITE_DIR}-${TARGET_DIR}-%Y%m%d-%H.%M
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
57 - create trash folder
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
58 - check if file in "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}/ fulfills [a-f0-9]{32}\..* or is a folder
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
59 - move to trash if is trash
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
60 - check if file is contained in ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-list
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
61 - if is not, move to trash
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
62 - Check_Files
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
63 - if not ISNEW (empty target folder):
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
64 - if not NOCLEAN (not skipping cleanup:
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
65 - Call Clean_Repository
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
66 - chdir to target folder
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
67 - TODO: chdir-free operation
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
68 - empty ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-err
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
69 - check file in current directory (${BASE_DIR}/${SITE_DIR}/${TARGET_DIR})
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
70 - skip if not correct file ([a-f0-9]{32}\..*)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
71 - put into ${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-error for every error files
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
72 - remove the files
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
73 - chdir to temp folder
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
74 - list ${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}, compare with error, exclude the errors, put into ${SITE_DIR}-${TARGET_DIR}-ok
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
75 - get list of new files - compare with ${SITE_DIR}-${TARGET_DIR}-list
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
76 - TODO: remove ls, grep -f dependencies
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
77 - if ISQUICK: skip check
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
78 - if not ISQUICK: print 'empty repository'
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
79 - copy ${SITE_DIR}-${TARGET_DIR}-list to ${SITE_DIR}-${TARGET_DIR}-newlist
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
80 - Fetch_Images
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
81 - chdir to ${BASE_DIR}/temp
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
82 - check if ${SITE_DIR}-${TARGET_DIR}-newlist is empty -> stop
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
83 - chdir to ${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
84 - start wget: wget -e continue=on -bi "${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}-newlist" -o "${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}.log"
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
85 - Init
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
86 - Add path (PATH)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
87 - Check command (fetch/check/quickfetch/* - JOB)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
88 - Check site (-s <site> or default - SITE)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
89 - Check if not clean folder (-nc/no clean - NOCLEAN)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
90 - Get tags (TAGS)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
91 - Check site - if SITE empty then set default (SITE=DEFAULT_SITE)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
92 - TODO: Validate SITE
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
93 - Check tag - if TAGS empty then Err_Fatal
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
94 - Get BASE_DIR: default to PWD - fallback to HOME
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
95 - Validate BASE_DIR: must absolute path
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
96 - Get TARGET_DIR: escape TAGS (replace / with _)
|
edhoprima@gmail.com <edhoprima@gmail.com>
parents:
diff
changeset
|
97 - Get SITE_DIR: escape SITE (remove ending /, replace / with _)
|