Fli
08-02-2015, 02:17 PM
Bash script to check Artists Against 419 and CLEAN MX .de webpages against websites hosted on local Linux Apache webserver & report via email if any is hosted.
No guarantee provided that this script will work or will not damage anything on the server. For me worked.
1. On the webserver, create folder /root/scamcheck/
then inside, create file /root/scamcheck/webpagestoextract
which will contain following 2 lines:
http://db.aa419.org/fakebankslist.php?psearch=YOURWEBSERVERIP
http://support.clean-mx.de/clean-mx/portals.php?ip=YOURWEBSERVERIP&sort=id%20DESC&response=alive
Then create another file: /root/scamcheck/scamcheck
with content:
# This script working for me to check webpages that lists fake websites.
# Then it check server apache virtualhost file to discover if any of the mentioned sites is not hosted on local server.
# If is hosted, then notiffication is sent to an email address.
# i saved this script as /root/scamcheck/scamcheck
# then create file /root/scamcheck/webpagestoextract which contains URLs to be checked against scam sites, one url per line.
# setup cronjob like:
# */10 * * * * /bin/sh /root/scamcheck/scamcheck
webpagestoextract=webpagestoextract
suspiciousdomains=suspiciousdomains
suspiciousdomainshosted=suspiciousdomainshosted
hosteddomains=$(cat /etc/httpd/conf/httpd.conf | grep "ServerName" | tr -d " " | sed -e "s/ServerName//g")
thisscriptdir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
[email protected]
# Empty suspiciousdomainshosted file if its not younger 4 months (is older), so if abuser start hosting same domain, im alerted again (number is in hours)
find $thisscriptdir -name "suspiciousdomainshosted" ! -ctime -2880 -delete
for webpage in $(cat $thisscriptdir/webpagestoextract);do
#echo "-------------------------------------------------------------------------------------------"
#echo "Download webpage $webpage content"
#echo "Check if any of listed domains (suspicious domains) on that page are hosted on this server"
#echo "-------------------------------------------------------------------------------------------"
curl --silent "$webpage" | grep -ahoP 'http[-a-zA-Z0-9@:%_\+.~#?&//=]{2,256}\.[a-z]{2,4}\b(\/[-a-zA-Z0-9@:%_\+.~#?&//=]*)?' | grep -v "419" | grep -v "w3." | awk -F/ '{print $3}' | sed -e "s/http:\/\/www.//g" | sed -e "s/www.//g" | sed -e "s/http:\/\///g" | sort -u > $suspiciousdomains > /dev/null 2>&1
#echo "Extracted suspicious domains:"
#cat $suspiciousdomains
#echo ""
for suspiciousdomain in $(cat $suspiciousdomains);do
# is suspicious domain $suspiciousdomain hosted?
if [[ "$hosteddomains" == *"$suspiciousdomain"* ]];then
# it is hosted. we already have it in suspiciousdomainshosted file (already been reported)?
if [[ "$(cat $thisscriptdir/$suspiciousdomainshosted)" != *"$suspiciousdomain"* ]];then
# this suspicious domain was not on file so its new! lets add it to the file and send an email alert
echo "$suspiciousdomain" >> suspiciousdomainshosted && sort -u suspiciousdomainshosted
#echo "Suspicious domains hosted:"
#cat suspiciousdomainshosted
#echo ""
echo "New suspicious domain hosted on $(hostname). Source: $webpage
$suspiciousdomain
Check this domain and suspend it from hosting billing system if needed. This is an email from "scamcheck" script." | mail -s "New suspicious domain at $(hostname)" $adminmail
fi
fi
done
done
# echo "Done, if no output, it means no new suspicious domain that is hosted at this server."
Then if you are sure above script is working, chmod the script file 700, run it if you really want to (maybe with "set -ex" on the first line to test it) and setup a conjob like:
*/10 * * * * /bin/sh /root/scamcheck/scamcheck
the script will run every 10 minutes to check if you do not host any scam webpages.
No guarantee provided that this script will work or will not damage anything on the server. For me worked.
1. On the webserver, create folder /root/scamcheck/
then inside, create file /root/scamcheck/webpagestoextract
which will contain following 2 lines:
http://db.aa419.org/fakebankslist.php?psearch=YOURWEBSERVERIP
http://support.clean-mx.de/clean-mx/portals.php?ip=YOURWEBSERVERIP&sort=id%20DESC&response=alive
Then create another file: /root/scamcheck/scamcheck
with content:
# This script working for me to check webpages that lists fake websites.
# Then it check server apache virtualhost file to discover if any of the mentioned sites is not hosted on local server.
# If is hosted, then notiffication is sent to an email address.
# i saved this script as /root/scamcheck/scamcheck
# then create file /root/scamcheck/webpagestoextract which contains URLs to be checked against scam sites, one url per line.
# setup cronjob like:
# */10 * * * * /bin/sh /root/scamcheck/scamcheck
webpagestoextract=webpagestoextract
suspiciousdomains=suspiciousdomains
suspiciousdomainshosted=suspiciousdomainshosted
hosteddomains=$(cat /etc/httpd/conf/httpd.conf | grep "ServerName" | tr -d " " | sed -e "s/ServerName//g")
thisscriptdir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
[email protected]
# Empty suspiciousdomainshosted file if its not younger 4 months (is older), so if abuser start hosting same domain, im alerted again (number is in hours)
find $thisscriptdir -name "suspiciousdomainshosted" ! -ctime -2880 -delete
for webpage in $(cat $thisscriptdir/webpagestoextract);do
#echo "-------------------------------------------------------------------------------------------"
#echo "Download webpage $webpage content"
#echo "Check if any of listed domains (suspicious domains) on that page are hosted on this server"
#echo "-------------------------------------------------------------------------------------------"
curl --silent "$webpage" | grep -ahoP 'http[-a-zA-Z0-9@:%_\+.~#?&//=]{2,256}\.[a-z]{2,4}\b(\/[-a-zA-Z0-9@:%_\+.~#?&//=]*)?' | grep -v "419" | grep -v "w3." | awk -F/ '{print $3}' | sed -e "s/http:\/\/www.//g" | sed -e "s/www.//g" | sed -e "s/http:\/\///g" | sort -u > $suspiciousdomains > /dev/null 2>&1
#echo "Extracted suspicious domains:"
#cat $suspiciousdomains
#echo ""
for suspiciousdomain in $(cat $suspiciousdomains);do
# is suspicious domain $suspiciousdomain hosted?
if [[ "$hosteddomains" == *"$suspiciousdomain"* ]];then
# it is hosted. we already have it in suspiciousdomainshosted file (already been reported)?
if [[ "$(cat $thisscriptdir/$suspiciousdomainshosted)" != *"$suspiciousdomain"* ]];then
# this suspicious domain was not on file so its new! lets add it to the file and send an email alert
echo "$suspiciousdomain" >> suspiciousdomainshosted && sort -u suspiciousdomainshosted
#echo "Suspicious domains hosted:"
#cat suspiciousdomainshosted
#echo ""
echo "New suspicious domain hosted on $(hostname). Source: $webpage
$suspiciousdomain
Check this domain and suspend it from hosting billing system if needed. This is an email from "scamcheck" script." | mail -s "New suspicious domain at $(hostname)" $adminmail
fi
fi
done
done
# echo "Done, if no output, it means no new suspicious domain that is hosted at this server."
Then if you are sure above script is working, chmod the script file 700, run it if you really want to (maybe with "set -ex" on the first line to test it) and setup a conjob like:
*/10 * * * * /bin/sh /root/scamcheck/scamcheck
the script will run every 10 minutes to check if you do not host any scam webpages.