Check Site Alive Wappalyzer

(补充：) 输入法坏了。所以用英文

refer to: https://github.com/wappalyzer/wappalyzer

https://security.stackexchange.com/questions/117131/how-to-find-out-what-programming-language-a-website-is-built-in

very good~

register ( no need ... )

9$ / month

github:

git clone https://github.com/wappalyzer/wappalyzer.git
cd wappalyzer
yarn install
yarn run link

usage:

node src/drivers/npm/cli.js https://somesite.me

result:

do some test: ( run as rails script )

ENV['RAILS_ENV'] = ARGV.first || ENV['RAILS_ENV'] || 'production'
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
require 'rails'
require 'rubygems'
require 'httparty'

require 'csv'


puts "== install wappalyzer from: https://github.com/wappalyzer/wappalyzer"

WAPPALYZER = "node /workspace2/coding_tools/wappalyzer/src/drivers/npm/cli.js"

def run url, server
  command = "#{WAPPALYZER} https://#{url}"
  result = `#{command}`
  puts "== result: #{result.inspect}"
  puts "== result.urls: #{JSON.parse(result)['urls'].inspect}"
  key = JSON.parse(result)['urls'].keys[0]
  puts "== key is: #{key}"
  if JSON.parse(result)['urls'][key]['status'] < 400
    server.update wappalyzer_result: result
  else
    command = "#{WAPPALYZER} http://#{url}"
    result = `#{command}`
    server.update wappalyzer_result: "https is not available, 
" + result
  end
end

all_servers = Server.all.order('id desc')
all_servers.each_slice(10) do |servers|
  threads = []
  servers.each do |server|
    threads << Thread.new do
      puts "site: #{server.name}, id: #{server.id}"
      run server.name, server
    end
  end
  threads.each {|t| t.join }
  sleep 10
end

result looks very good:

some practice:  CPU may got high if run multiple threads

caution:

it tops when executing mass tasks. ( multiple thread ) 

maybe because this PC only has 4 core CPU