Net Scan: 4TB All, 40TB BAB&app_vars

[jspenguin2017]

20 packages scanned, 1 package processed, about 50,000+ packages expected per month.
About 80 GB data scanned out of about 200+ TB expected per month.

Hard anti-adblock:

# BAB
http://f4.motogon.ru/motokross/mxgp-2018/msg717172/
https://dc-chronicle.com/2018/08/01/robert-mueller-refers-democrat-tony-podesta-criminal-charges/
https://www.sofiotheque.info/2018/02/telecharger-50-dossiers-de-maladies.html
# Other
http://www.kizi.cm/    # Comes up every time

Annoyance:

# anOptions
http://allamericansthings.com/2018/11/prototyping-the-betentacled-inflatable-soft-robots-of-zero-gee/
http://thejewishvoice.com/2016/03/16/intel-acquires-replay-technologies-to-up-its-sports-game/
https://usa.watchpro.com/millennials-reason-behind-luxury-spending-boost-china/
http://www.ebizlatam.com/tag/gerardo-coronel/
http://www.fagagnaonline.com/
https://2spendless.com/?tag=forever
https://centrafriqueactu.com/2018/11/01/congo-la-gouvernance-forestiere-au-coeur-dun-forum-regional-a-brazzaville/
https://www.betglob.pl/tag/33

[jspenguin2017]

This scanner is just an initial prototype, I hacked it together within 30 minutes, I'll write proper automation scripts later.

[okiehsch]

I can't consistently reproduce the BAB entries, never had that issue.

[okiehsch]

ebizlatam.com is fixed in the regional list.

[okiehsch]

@jspenguin2017 let's use this issue as reference for your future scan results.

[jspenguin2017]

OK, I'll close this for now, I'll reopen when new data become available.

The scanner currently flags BAB, anOptions, and app_vars, I definitely want to add more. Once I start spinning up clusters to boost the scanning speed, it'll emit thousands and thousands lines of result, I'm thinking about make the scanner emit filter rules directly, honestly I don't want to manually verify every result.

[jspenguin2017]

These are from 20 packages, there's over 50000 left. As this pace, there'll be over 2500 times more... I need to think of a better strategy.

[okiehsch]

Well, atleast one site includes multiple anti-adblock scripts, I will fix such sites seperately and reference this issue.

http://multicoinfaucet.cf/doge/?cc=reCaptcha

[jspenguin2017]

I scanned 30 more packages, but only found 4 BAB, looks like my initial estimate was way off.

Also, I don't know whether the domains will repeat in later packages. I need to update my scanner to keep track of it. I also want my scanner to be able to handle AdFly, IL, and more.

On top of this, I need to find a way to drain the backlog, so I'm shutting down my server for now.

[okiehsch]

http://paytoshi.in/?cc=reCaptcha I can reproduce a popup, no anti-adblock message on my end.

[jspenguin2017]

Yes, only popup. Closing for now, I need some time to think about the next step.

[smed79]

no anti-adblock message on my end.

Go to http://paytoshi.in/?cc=reCaptcha
Click "You are Human".
The page will redirect you to http://payco.xyz/FButmCYnOojY ==> anti adblock

[okiehsch]

I worked through the first 100 lines of net-scan-backlog/anOptions0.txt and net-scan-backlog/anOptions1.txt.

2spendless.com
allamericansthings.com
bookwonderland.com
dailynexus.com
centrafriqueactu.com
ebizlatam.com
en-contact.com
homasg.com
kizi.cm
lentes-et-poux.fr
moonbunnycafe.com
salient.org.nz
socialbookmarkings.us
softmaker.kz
thejewishvoice.com
usa.watchpro.com
vodoleyworld.ru

are either already fixed, can't reproduce the anti-adblock messsage, can't connect or parked domains.
currentaffairs.gktoday.in uses admiral.

I will add the rest.

[jspenguin2017]

18 out of 200, that's not an acceptable ratio, I'll definitely need to find a way to automate it, at least partially.

[jspenguin2017]

OK, I finally found a working tutorial for VNC: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-vnc-on-debian-9

Update: It worked perfectly fine on my dev device, I deploy it, and...

[jspenguin2017]

@okiehsch It's working! Be ready for good news soon 😄

Update:
It's about half way done scanning those 1049 entries, hopefully nothing will go wrong and it'll finish tomorrow morning.

[reza41]

3d eart scanner