Your Ad can be there
+ Post New Thread
Results 1 to 2 of 2

Thread: Linux bash script to check if we do not host scams listed on Artists Against 419...

  1. #1
    Administrator
    Join Date
    Mar 2013
    Posts
    2,725

    Linux bash script to check if we do not host scams listed on Artists Against 419...

    Bash script to check Artists Against 419 and CLEAN MX .de webpages against websites hosted on local Linux Apache webserver & report via email if any is hosted.

    No guarantee provided that this script will work or will not damage anything on the server. For me worked.


    1. On the webserver, create folder /root/scamcheck/

    then inside, create file /root/scamcheck/webpagestoextract
    which will contain following 2 lines:

    Then create another file: /root/scamcheck/scamcheck
    with content:

    # This script working for me to check webpages that lists fake websites.
    # Then it check server apache virtualhost file to discover if any of the mentioned sites is not hosted on local server.
    # If is hosted, then notiffication is sent to an email address.

    # i saved this script as /root/scamcheck/scamcheck

    # then create file /root/scamcheck/webpagestoextract which contains URLs to be checked against scam sites, one url per line.

    # setup cronjob like:
    # */10 * * * * /bin/sh /root/scamcheck/scamcheck

    webpagestoextract=webpagestoextract
    suspiciousdomains=suspiciousdomains
    suspiciousdomainshosted=suspiciousdomainshosted
    hosteddomains=$(cat /etc/httpd/conf/httpd.conf | grep "ServerName" | tr -d " " | sed -e "s/ServerName//g")
    thisscriptdir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
    adminmail=[email protected]

    # Empty suspiciousdomainshosted file if its not younger 4 months (is older), so if abuser start hosting same domain, im alerted again (number is in hours)
    find $thisscriptdir -name "suspiciousdomainshosted" ! -ctime -2880 -delete

    for webpage in $(cat $thisscriptdir/webpagestoextract);do
    #echo "-------------------------------------------------------------------------------------------"
    #echo "Download webpage $webpage content"
    #echo "Check if any of listed domains (suspicious domains) on that page are hosted on this server"
    #echo "-------------------------------------------------------------------------------------------"

    curl --silent "$webpage" | grep -ahoP 'http[-a-zA-Z0-9@:%_\+.~#?&//=]{2,256}\.[a-z]{2,4}\b(\/[-a-zA-Z0-9@:%_\+.~#?&//=]*)?' | grep -v "419" | grep -v "w3." | awk -F/ '{print $3}' | sed -e "s/http:\/\/www.//g" | sed -e "s/www.//g" | sed -e "s/http:\/\///g" | sort -u > $suspiciousdomains > /dev/null 2>&1

    #echo "Extracted suspicious domains:"
    #cat $suspiciousdomains
    #echo ""

    for suspiciousdomain in $(cat $suspiciousdomains);do
    # is suspicious domain $suspiciousdomain hosted?
    if [[ "$hosteddomains" == *"$suspiciousdomain"* ]];then
    # it is hosted. we already have it in suspiciousdomainshosted file (already been reported)?
    if [[ "$(cat $thisscriptdir/$suspiciousdomainshosted)" != *"$suspiciousdomain"* ]];then
    # this suspicious domain was not on file so its new! lets add it to the file and send an email alert
    echo "$suspiciousdomain" >> suspiciousdomainshosted && sort -u suspiciousdomainshosted

    #echo "Suspicious domains hosted:"
    #cat suspiciousdomainshosted
    #echo ""
    echo "New suspicious domain hosted on $(hostname). Source: $webpage

    $suspiciousdomain

    Check this domain and suspend it from hosting billing system if needed. This is an email from "scamcheck" script." | mail -s "New suspicious domain at $(hostname)" $adminmail
    fi
    fi
    done
    done
    # echo "Done, if no output, it means no new suspicious domain that is hosted at this server."
    Then if you are sure above script is working, chmod the script file 700, run it if you really want to (maybe with "set -ex" on the first line to test it) and setup a conjob like:
    */10 * * * * /bin/sh /root/scamcheck/scamcheck

    the script will run every 10 minutes to check if you do not host any scam webpages.

  2. #2


    Is this useful / helpfull? Yes | No
    A great initiative. Thank you. Yes, and it beats monitoring a RSS and hoping you don't miss out in a busy period.

    For aa419, a more correct approach would be:
    Similarly, if you're a network owner, you could check for your ASN, say 22612:
    This would allow you the benefit of only checking for active websites, you don't need to disable disabled websites (or those on hold,but you could monitor for holds similarly)

    This should only be needed once or twice a day. Obviously the website admins get a bit testy if you monitor 5 times a minute of like

+ Post New Thread

Similar Threads

  1. Replies: 2
    Last Post: 08-05-2020, 12:29 PM
  2. Replies: 0
    Last Post: 05-31-2015, 03:16 PM
  3. Replies: 0
    Last Post: 11-30-2014, 04:19 PM
  4. WTS Linux bash script automation service
    By Fli in forum WTS/WTB Programming services
    Replies: 0
    Last Post: 11-08-2013, 11:51 AM
  5. Linux bash script for easy IP tables IP inserting / deleting
    By Fli in forum Programming, scripting
    Replies: 0
    Last Post: 09-27-2013, 09:04 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
 Protected by : ZB BLOCK  &  StopForumSpam