annotate moefetch.sh @ 227:8b1f6f6b6a3b

Bugfixes: - Login and password are not always used now. Added new global variable to check it. - 'export PATH' is now only done if additional path(s) specified. Not really enhancements: - New coding style: - Local variables now start with double underscores and global variables start with single underscore. Both lower letter. - Function name is now snake_case. Camel_And_Snake_Case doesn't actually make sense :> - More comments. Still need to add even more comments.
author edhoprima@gmail.com <edhoprima@gmail.com>
date Mon, 14 Dec 2009 09:57:35 +0000
parents f8be4a3d3b4a
children 5d3a0645b504
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
221
e891b563b797 wrong rule caused mass headache
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 220
diff changeset
1 #!/bin/sh
148
edhoprima
parents:
diff changeset
2
edhoprima
parents:
diff changeset
3 # Copyright (c) 2009, edogawaconan <me@myconan.net>
edhoprima
parents:
diff changeset
4 #
edhoprima
parents:
diff changeset
5 # Permission to use, copy, modify, and/or distribute this software for any
edhoprima
parents:
diff changeset
6 # purpose with or without fee is hereby granted, provided that the above
edhoprima
parents:
diff changeset
7 # copyright notice and this permission notice appear in all copies.
edhoprima
parents:
diff changeset
8 #
edhoprima
parents:
diff changeset
9 # THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
edhoprima
parents:
diff changeset
10 # WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
edhoprima
parents:
diff changeset
11 # MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
edhoprima
parents:
diff changeset
12 # ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
edhoprima
parents:
diff changeset
13 # WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
edhoprima
parents:
diff changeset
14 # ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
edhoprima
parents:
diff changeset
15 # OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
16 #
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
17 # Lots of bugs here. Use with care
148
edhoprima
parents:
diff changeset
18 # USE WITH CARE
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
19 #
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
20 # what it does: fetch every picture that has the specified TAGS.
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
21 # requirement: wget, libxslt, openssl
148
edhoprima
parents:
diff changeset
22
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
23 # program additional paths for: cut, sed, wc, openssl, wget, xsltproc, grep
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
24 ADDITIONAL_PATH=
148
edhoprima
parents:
diff changeset
25
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
26 # default server address. Danbooru only! I do not take responsibility of stupidity.
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
27 DEFAULT_SITE="moe.imouto.org"
148
edhoprima
parents:
diff changeset
28
edhoprima
parents:
diff changeset
29 # base directory. make sure it's writeable. I do not take responsibility if you don't own the folder and files as no check is done for this one.
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
30 # Structure is ${BASE_DIR}/<TAGS>
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
31 # Absolute path only.
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
32 # Leave empty to use whatever folder you're running this at
193
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
33 BASE_DIR=
148
edhoprima
parents:
diff changeset
34
edhoprima
parents:
diff changeset
35 # not user modifiable from here
edhoprima
parents:
diff changeset
36
193
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
37 ### TODO:
211
fbb93733098e update to 0.2
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 210
diff changeset
38 ### - sanity validator(?)
193
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
39 ### - unified repository to save bandwidth
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
40 ### - bug stomping
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
41 ### - sanity checking
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
42 ### - MOAR comments
193
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
43 ### WILL BE FOR 0.3
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
44
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
45 # useless welcome message. Also version
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
46 Msg_Welcome() {
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
47 MOEFETCHVERSION="0.3-beta2"
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
48 echo "moefetch ${MOEFETCHVERSION}
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
49 Copyright (c) 2009 edogawaconan <me@myconan.net>
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
50 "
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
51 }
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
52
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
53 get_md5() { cat -- "$1" | openssl dgst -md5; }
226
f8be4a3d3b4a - Fix basename to escape with / instead of -- (workaround for conflict between BSD/GNU/Solaris implementations)
edhoprima
parents: 225
diff changeset
54 get_basename() { basename /"$1"; }
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
55 get_filename() { get_basename "${1%.*}"; }
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
56 get_cleantags() { printf "%s " "$@" | sed -e 's/\&/%26/g;s/=/%3D/g'; }
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
57 Is_NotMD5() { get_filename "$1" | sed -e 's/\([0-9a-f]\{32\}\)//g'; }
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
58
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
59
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
60 # fatal error handler
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
61 Err_Fatal() {
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
62 echo "
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
63 Fatal error: ${1}"
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
64 exit 1
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
65 }
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
66
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
67 Err_Impossible() {
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
68 echo "
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
69 Impossible error. Or you modified content of the working directories when the script is running.
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
70 Please report to moefetch.googlecode.com if you see this message (complete with entire run log)"
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
71 exit 1
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
72 }
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
73
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
74 # help message
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
75 Err_Help() {
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
76 echo "moefetch.sh COMMAND [-s SITE_URL] TAGS
174
0948e76a57a1 added help. Bump to 0.1-beta2
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 173
diff changeset
77
176
3d2ae9417273 even more improvement
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 175
diff changeset
78 COMMAND:
3d2ae9417273 even more improvement
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 175
diff changeset
79 (quick)fetch: do a complete update. Add prefix quick to skip file checking
3d2ae9417273 even more improvement
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 175
diff changeset
80 check: get list of new files, clean up local folder and print total new files
175
5b7a154dbd21 cosmetics fix for help message
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 174
diff changeset
81
177
4d7de9c3b98c whoops typo
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 176
diff changeset
82 -s SITE_URL: Specify URL of the Danbooru powered site you want to leech from. Default is ${DEFAULT_SITE}
175
5b7a154dbd21 cosmetics fix for help message
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 174
diff changeset
83
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
84 TAGS: Tags you want to download. Separated by spaces. Tag name follows standard Danbooru tagging scheme"
193
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
85 exit 2
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
86 }
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
87
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
88 # generate link by transforming xml
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
89 Generate_Link() {
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
90 echo "
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
91 Fetching XML file"
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
92 tempnum=1000
224
0ac1805621d4 fix for FreeBSD
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 223
diff changeset
93 iternum=1
195
652d9e268cee test migration to printf
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 194
diff changeset
94 > "${TEMP_PREFIX}-list"
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
95 while [ "${tempnum}" -ge 1000 ]; do
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
96 __url="http://${SITE}/post/index.xml?tags=$(get_cleantags "${TAGS}")&offset=0&limit=1000&page=${iternum}"
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
97 [ ${_use_login} -eq 1 ] && __url="${__url}&login=${LOGIN_USER}&password_hash=${LOGIN_PASS}"
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
98 wget "${__url}" -O "${TEMP_PREFIX}-xml" -e continue=off || Err_Fatal "Failed download catalog file"
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
99 printf "Processing XML file... "
213
dd95cf01602c working around limit
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 212
diff changeset
100 # xslt evilry
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
101 xsltproc - "${TEMP_PREFIX}-xml" <<EOF | sed 's/.*\(http.*\)\(\/[a-f0-9]\{32\}\).*\.\([^\.]*\)/\1\2.\3/g' | grep ^http > "${TEMP_PREFIX}-templist"
148
edhoprima
parents:
diff changeset
102 <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
edhoprima
parents:
diff changeset
103 <xsl:output method="xml" indent="yes"/>
edhoprima
parents:
diff changeset
104 <xsl:template match="post">
edhoprima
parents:
diff changeset
105 <xsl:value-of select="@file_url" />
edhoprima
parents:
diff changeset
106 </xsl:template>
edhoprima
parents:
diff changeset
107 </xsl:stylesheet>
edhoprima
parents:
diff changeset
108 EOF
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
109 tempnum=$(echo $(wc -l < "${TEMP_PREFIX}-templist"))
224
0ac1805621d4 fix for FreeBSD
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 223
diff changeset
110 iternum=$((iternum + 1))
213
dd95cf01602c working around limit
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 212
diff changeset
111 cat "${TEMP_PREFIX}-templist" >> "${TEMP_PREFIX}-list"
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
112 echo "${tempnum} file(s) available"
213
dd95cf01602c working around limit
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 212
diff changeset
113 done
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
114 numfiles=$(echo $(wc -l < "${TEMP_PREFIX}-list"))
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
115 echo "${numfiles} file(s) available on server"
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
116 [ "${numfiles}" -gt 0 ] || Err_Fatal "Error in processing list or no files can be found with specified tag(s) or site."
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
117 }
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
118
148
edhoprima
parents:
diff changeset
119
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
120 Progress_Init() {
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
121 _last="-"
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
122 printf "${_last}"
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
123 }
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
124
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
125 Progress_Anim() {
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
126 case "${_last}" in
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
127 /) _last="-";;
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
128 -) _last=\\;;
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
129 \\) _last=\|;;
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
130 \|) _last="/";;
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
131 esac
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
132 printf "\b${_last}"
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
133 }
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
134
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
135 Progress_Done() { printf "\bdone\n"; }
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
136
200
8efa600ebfdb purge ls
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 199
diff changeset
137 # getting rid of ls (as per suggestion)
8efa600ebfdb purge ls
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 199
diff changeset
138 Count_Files() {
224
0ac1805621d4 fix for FreeBSD
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 223
diff changeset
139 numfiles=0
0ac1805621d4 fix for FreeBSD
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 223
diff changeset
140 for dircontent in "${*}/"* "${*}/".*; do
0ac1805621d4 fix for FreeBSD
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 223
diff changeset
141 if [ "${dircontent}" != "${*}/*" ] || [ -e "${dircontent}" ]; then
0ac1805621d4 fix for FreeBSD
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 223
diff changeset
142 numfiles=$((numfiles + 1))
200
8efa600ebfdb purge ls
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 199
diff changeset
143 fi
8efa600ebfdb purge ls
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 199
diff changeset
144 done
224
0ac1805621d4 fix for FreeBSD
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 223
diff changeset
145 echo $((numfiles - 2))
200
8efa600ebfdb purge ls
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 199
diff changeset
146 }
8efa600ebfdb purge ls
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 199
diff changeset
147
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
148 # check tools availability
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
149 Check_Tools() {
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
150 # verify all programs required do indeed exist
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
151 commands="cut sed wc wget xsltproc xargs rm mkdir chown comm grep date openssl"
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
152 for cmd in ${commands}
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
153 do
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
154 [ "$(command -v "${cmd}")" ] || Err_Fatal "${cmd} doesn't exist in ${PATH}"
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
155 done
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
156 }
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
157
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
158 # verify required folders exist and writeable
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
159 Check_Folders(){
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
160 [ -O "${BASE_DIR}" ] || Err_Fatal "You don't own ${BASE_DIR}. Please fix ${BASE_DIR} or run this script in your own directory."
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
161 for directory in temp trash deleted "${SITE_DIR}/${TARGET_DIR}"; do
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
162 if [ ! -d "${BASE_DIR}/${directory}" ]; then
216
a869987c4646 did I say 'mess up'?
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 215
diff changeset
163 mkdir -p "${BASE_DIR}/${directory}" || Err_Impossible
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
164 fi
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
165 if [ ! -O "${BASE_DIR}/${directory}" ]; then
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
166 echo "You don't own the ${BASE_DIR}/${directory}, applying globally writeable permission on it"
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
167 chmod -R u=rwX,g=rwX,o=rwX "${BASE_DIR}/${directory}" || Err_Impossible
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
168 fi
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
169 done
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
170 [ "$(Count_Files "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}")" -eq 0 ] && ISNEW=1
201
30d2fb656029 scrapping grep -vf
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 200
diff changeset
171 for i in error ok list newlist templist; do
196
4d28f3a957ee cleanup~
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 195
diff changeset
172 touch "${TEMP_PREFIX}-${i}" || Fatal_Err "Error creating ${TEMP_PREFIX}-${i}. This shouldn't happen"
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
173 done
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
174 #
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
175 }
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
176
187
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
177 # Do some cleanup
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
178 Cleanup_Repository() {
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
179 # current dir: ${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}
207
17d816a63b4c final progress version
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 206
diff changeset
180 printf "Cleaning up repository folder... "
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
181 Progress_Init
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
182 trash_dir="${BASE_DIR}/trash/${trash_dir}/$(date -u "+${SITE_DIR}-${TARGET_DIR}-%Y%m%d-%H.%M")"
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
183 trashes="These files have been moved to ${trash_dir}:"
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
184 has_trash=
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
185 if [ ! -d "${trash_dir}" ]; then
216
a869987c4646 did I say 'mess up'?
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 215
diff changeset
186 mkdir -p "${trash_dir}" || Err_Impossible
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
187 else
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
188 if [ ! -O "${trash_dir}" ]; then
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
189 chmod -R u=rwX,g=rwX,o=rwX "${BASE_DIR}/${directory}" || Err_Impossible
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
190 fi
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
191 fi
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
192 for trash in "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}/"*
187
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
193 do
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
194 is_trash=
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
195 if [ -d "${trash}" ] || [ -n "$(Is_NotMD5 "${trash}")" ] || [ -z "$(grep "$(get_basename "${trash}")" "${TEMP_PREFIX}-list")" ]; then
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
196 is_trash=1
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
197 has_trash=1
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
198 mv -f -- "${trash}" "${trash_dir}" || Err_Impossible
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
199 trashes="${trashes}
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
200 $(get_basename "${trash}")"
187
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
201 fi
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
202 Progress_Anim
187
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
203 done
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
204 rmdir "${trash_dir}" 2>/dev/null
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
205 Progress_Done
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
206 [ -n "${has_trash}" ] && echo "${trashes}"
187
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
207 }
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
208
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
209 # check files correctness
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
210 Check_Files() {
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
211 if [ ! -n "${ISNEW}" ]; then
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
212 [ -z "${NOCLEAN}" ] && Cleanup_Repository
207
17d816a63b4c final progress version
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 206
diff changeset
213 printf "Checking for errors... "
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
214 Progress_Init
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
215 files_error="These files do not match its md5:"
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
216 files_notdanbooru="These files are not checked:"
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
217 has_err_filename=
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
218 has_err_md5=
196
4d28f3a957ee cleanup~
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 195
diff changeset
219 > "${TEMP_PREFIX}-error"
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
220 > "${TEMP_PREFIX}-ok"
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
221 for file in "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}/"*
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
222 do
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
223 if [ "${file}" != "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}/*" ]; then
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
224 if [ -n "$(Is_NotMD5 "${file}")" ] || [ -d "${file}" ]; then
217
77cd21d714f6 screwup fix
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 216
diff changeset
225 files_notdanbooru="${files_notdanbooru}
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
226 $(get_basename "${file}")"
217
77cd21d714f6 screwup fix
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 216
diff changeset
227 has_err_filename=1
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
228 else
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
229 if [ "$(get_md5 "${file}")" = "$(get_filename "${file}")" ]; then
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
230 echo "$(get_basename "${file}")" >> "${TEMP_PREFIX}-ok"
217
77cd21d714f6 screwup fix
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 216
diff changeset
231 else
77cd21d714f6 screwup fix
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 216
diff changeset
232 rm "${file}" || Err_Fatal "Error removing ${file}"
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
233 echo "$(get_basename "${file}")" >> "${TEMP_PREFIX}-error"
217
77cd21d714f6 screwup fix
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 216
diff changeset
234 files_error="${files_error}
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
235 $(get_basename "${file}")"
217
77cd21d714f6 screwup fix
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 216
diff changeset
236 has_err_md5=1
77cd21d714f6 screwup fix
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 216
diff changeset
237 fi
187
efd957294c8c refactoring. cleanup. etc.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 186
diff changeset
238 fi
148
edhoprima
parents:
diff changeset
239 fi
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
240 Progress_Anim
148
edhoprima
parents:
diff changeset
241 done
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
242 Progress_Done
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
243 if [ ! -n "${has_err_md5}" ] && [ ! -n "${has_err_filename}" ]; then
203
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 202
diff changeset
244 echo "All files OK"
170
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 169
diff changeset
245 else
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
246 if [ -n "${has_err_md5}" ]; then
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
247 echo "${files_error}"
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
248 echo "$(echo $(wc -l < "${TEMP_PREFIX}-error")) file(s) removed"
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
249 fi
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
250 [ -n "${has_err_filename}" ] && echo "${files_notdanbooru}"
170
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 169
diff changeset
251 fi
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
252 echo "$(echo $(wc -l < "${TEMP_PREFIX}-ok")) file(s) available locally"
148
edhoprima
parents:
diff changeset
253
207
17d816a63b4c final progress version
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 206
diff changeset
254 printf "Generating list of new files... "
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
255 Progress_Init
218
aeca29670e26 broken!
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 217
diff changeset
256 cp -f "${TEMP_PREFIX}-list" "${TEMP_PREFIX}-templist"
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
257 while read -r is_ok; do
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
258 grep -v "${is_ok}" "${TEMP_PREFIX}-templist" > "${TEMP_PREFIX}-newlist"
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
259 cp -f "${TEMP_PREFIX}-newlist" "${TEMP_PREFIX}-templist" || Err_Impossible
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
260 Progress_Anim
203
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 202
diff changeset
261 done < "${TEMP_PREFIX}-ok"
205
2e866999c042 now with useless animation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 204
diff changeset
262 Progress_Done
214
a6624fb9b317 major cleanup. tweaking.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 213
diff changeset
263 echo "$(echo $(wc -l < "${TEMP_PREFIX}-newlist")) file(s) to be downloaded"
148
edhoprima
parents:
diff changeset
264 else
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
265 if [ -n "${ISQUICK}" ]; then
207
17d816a63b4c final progress version
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 206
diff changeset
266 echo "Quick mode selected. Skipping check"
152
67df02877319 added quickfetch for skipping file checking
edhoprima
parents: 151
diff changeset
267 else
67df02877319 added quickfetch for skipping file checking
edhoprima
parents: 151
diff changeset
268 echo "Empty local repository"
67df02877319 added quickfetch for skipping file checking
edhoprima
parents: 151
diff changeset
269 fi
200
8efa600ebfdb purge ls
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 199
diff changeset
270 cat "${TEMP_PREFIX}-list" > "${TEMP_PREFIX}-newlist"
148
edhoprima
parents:
diff changeset
271 fi
edhoprima
parents:
diff changeset
272 }
edhoprima
parents:
diff changeset
273
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
274 # start downloading the images
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
275 Fetch_Images() {
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
276 if [ "$(echo $(wc -l < "${TEMP_PREFIX}-newlist"))" -eq 0 ]; then
148
edhoprima
parents:
diff changeset
277 echo "No new file"
edhoprima
parents:
diff changeset
278 else
207
17d816a63b4c final progress version
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 206
diff changeset
279 printf "Starting wget... "
160
68227a30d0b3 forgot to fix Fetch_Images to reflect new folder naming scheme
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 159
diff changeset
280 cd "${BASE_DIR}/${SITE_DIR}/${TARGET_DIR}"
196
4d28f3a957ee cleanup~
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 195
diff changeset
281 wget -e continue=on -bi "${TEMP_PREFIX}-newlist" -o "${TEMP_PREFIX}.log"
148
edhoprima
parents:
diff changeset
282 fi
edhoprima
parents:
diff changeset
283 }
edhoprima
parents:
diff changeset
284
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
285 # initialize base variables and initial command check
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
286 init()
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
287 {
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
288 # path initialization
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
289 # check if additional path is specified
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
290 if [ -n "${ADDITIONAL_PATH}" ]
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
291 then
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
292 # insert the additional path
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
293 PATH="${ADDITIONAL_PATH}:${PATH}"
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
294 export PATH
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
295 fi
158
cba73f6a96bb grep check. OpenSolaris' default grep doesn't support -f
edhoprima
parents: 157
diff changeset
296
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
297 # misc variables
166
cc60e8cf7793 dunno :<
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 165
diff changeset
298 ISQUICK=
cc60e8cf7793 dunno :<
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 165
diff changeset
299 ISNEW=
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
300 # variable to check whether a login is used or not
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
301 _use_login=0
215
710082ce6788 major cleanup part2.
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 214
diff changeset
302
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
303 # minimum number of arguments: 2 (command and tag). If less than two, exit and print help message
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
304 [ $# -lt 2 ] && Err_Help
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
305 case "$1" in
174
0948e76a57a1 added help. Bump to 0.1-beta2
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 173
diff changeset
306 check|fetch|quickfetch)
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
307 echo "Starting..."
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
308 JOB="$1"
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
309 ;;
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
310 *)
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
311 Err_Help
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
312 ;;
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
313 esac
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
314 shift
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
315 SITE=
185
6d926d4b3c5a initial clean system support
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 184
diff changeset
316 TAGS=
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
317 __has_pass=0
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
318 __has_user=0
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
319 while getopts "s:(site)n(noclean)u:(user)p:(password)" opt
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
320 do
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
321 case "$opt" in
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
322 s) SITE="$OPTARG";;
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
323 n) NOCLEAN=1;;
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
324 p)
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
325 LOGIN_PASS=$(printf "%s" "$OPTARG" | openssl dgst -sha1)
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
326 __has_pass=1
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
327 ;;
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
328 u)
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
329 LOGIN_USER="$OPTARG"
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
330 __has_user=1
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
331 ;;
185
6d926d4b3c5a initial clean system support
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 184
diff changeset
332 esac
6d926d4b3c5a initial clean system support
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 184
diff changeset
333 done
225
265a9ca47a19 - Replaced md5(sum) with openssl. Less platform dependent because the tool is same across platforms
edhoprima
parents: 224
diff changeset
334 TAGS="$@"
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
335 [ -n "${SITE}" ] || SITE=${DEFAULT_SITE}
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
336 [ -n "${TAGS}" ] || Err_Fatal "No tag specified"
181
d3b7927bdb2b restructuring and add check if the xml is processed properly
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 180
diff changeset
337 # Get base folder - default, current folder or fallback to ${HOME}
223
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
338 [ -n "${BASE_DIR}" ] || BASE_DIR=${PWD}
04ad0b0a3c63 revert back to [
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 222
diff changeset
339 [ -n "${BASE_DIR}" ] || BASE_DIR=${HOME}
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
340 [ -n "$(echo "${BASE_DIR}" | cut -c1 | grep \/)" ] || BASE_DIR="/${BASE_DIR}"
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
341 # see if both pass and use are set. If they're set, switch _use_login variable content to 1.
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
342 [ ${__has_pass} -eq 1 -a ${__has_user} -eq 1 ] && _use_login=1
181
d3b7927bdb2b restructuring and add check if the xml is processed properly
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 180
diff changeset
343
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
344 echo "Tags: ${TAGS}"
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
345 # slash is not wanted for folder name
193
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
346 TARGET_DIR=$(echo "${TAGS}" | sed -e 's/\//_/g')
ac6533a8fb51 - Documentation
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 192
diff changeset
347 SITE_DIR=$(echo "${SITE}" | sed -e 's/\/$//g;s/\//_/g')
195
652d9e268cee test migration to printf
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 194
diff changeset
348 TEMP_PREFIX="${BASE_DIR}/temp/${SITE_DIR}-${TARGET_DIR}"
159
75fe19903b74 Major cleanup
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 158
diff changeset
349 }
148
edhoprima
parents:
diff changeset
350
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
351 main()
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
352 {
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
353 # initialization
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
354 Msg_Welcome
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
355 init "$@"
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
356 Check_Tools
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
357 Check_Folders
158
cba73f6a96bb grep check. OpenSolaris' default grep doesn't support -f
edhoprima
parents: 157
diff changeset
358
148
edhoprima
parents:
diff changeset
359
227
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
360 # let's do the job!
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
361 case "${JOB}" in
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
362 check)
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
363 Generate_Link
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
364 Check_Files
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
365 ;;
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
366 fetch)
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
367 Generate_Link
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
368 Check_Files
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
369 Fetch_Images
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
370 ;;
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
371 quickfetch)
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
372 ISNEW=1
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
373 ISQUICK=1
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
374 Generate_Link
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
375 Check_Files
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
376 Fetch_Images
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
377 ;;
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
378 esac
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
379 }
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
380
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
381 # call the main routine!
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
382 main "$@"
8b1f6f6b6a3b Bugfixes:
edhoprima@gmail.com <edhoprima@gmail.com>
parents: 226
diff changeset
383