Public
Authored by Jiří Helebrant

omnibar check

instalace zavislosti:

$ npm init -y
$ npm install puppeteer

seznam url:

$ cat list.txt
https://www.nic.cz/manager/
http://dns-tester.cz
https://de.eidasnode.cz/eidas-middleware/Metadata
http://kdomispraviauto.cz
https://bird.network.cz
https://piwik.nic.cz
https://yangson.labs.nic.cz

nakrmeni skriptu seznamem:

$ node omnibarcheck.js list.txt > omnibar.json

vysledek (jsony oddelene \n):

$ cat omnibar.json
{"url":"https://www.nic.cz/manager/","finalUrl":"https://www.nic.cz/manager/login/?next=/manager/","included":false}
{"url":"http://dns-tester.cz","error":"Error: net::ERR_NAME_NOT_RESOLVED at http://dns-tester.cz"}
{"url":"https://de.eidasnode.cz/eidas-middleware/Metadata","finalUrl":"https://de.eidasnode.cz/eidas-middleware/Metadata","included":false}
{"url":"http://kdomispraviauto.cz","finalUrl":"https://www.dobradomena.cz/cs/co-je-to-domena/jake-je-nejlepsi-domenove-jmeno/automechanik-petr/","included":true,"isNg":true,"datafiles":["https://www.nic.cz/files/CORS/projects-bar/basic-datafile.json"],"flavor":"basic"}
{"url":"https://bird.network.cz","finalUrl":"https://bird.network.cz/","included":true,"isNg":false,"datafiles":["https://www.nic.cz/files/CORS/projects-bar/basic-datafile.json","https://www.nic.cz/files/CORS/projects-bar/geek-lista.js"],"flavor":"basic"}
{"url":"https://piwik.nic.cz","finalUrl":"https://piwik.nic.cz/","included":false}
{"url":"https://yangson.labs.nic.cz","finalUrl":"https://yangson.labs.nic.cz/","included":false}

hodnoty ve vysledcich:

{
  url: // string, url ze seznamu,
  finalUrl: // string, url na ktere prohlizec
            // po vsech presmerovanich skoncil
  included: // bool, zda je na strance omnibar
  isNg: // bool, true pokud je pouzit
        // https://gitlab.labs.nic.cz/websites/omnibar-ng
        // a ne ten stary
  datafiles: // pole stringu, skript zachycuje requesty
             // obsahujici "CORS/projects-bar" (coz by mely byt
             // datove soubory s odkazy) a jejich URL uklada sem
  flavor: // string, soubor odkazu ktere si omnibar nacetl, parsuje se z tech datafiles
          // (basic nebo geek, https://trac.nic.cz/wiki/devel/ginger/omnibar)
}

isNg, datafiles a flavor jsou tam samozrejme jen pokud je incluced: true

taky tam muze byt error, pokud se prohlizec z nejakeho duvodu (domena neni v DNS, nejaka chyba site apod.) nepripojil:

{"url":"http://dns-tester.cz","error":"Error: net::ERR_NAME_NOT_RESOLVED at http://dns-tester.cz"}

pomoci jq --slurp se da z jednotlivych jsonu vyrobit jedno velke pole objektu:

$ cat omnibar.json | jq --slurp
[
  {
    "url": "https://cert-cr.cz",
    "finalUrl": "https://cert-cr.cz/en/",
    "included": false
  },
  {
    "url": "https://csirt.cz",
    "finalUrl": "https://csirt.cz/en/",
    "included": true,
    "isNg": true,
    "datafiles": [
      "https://www.nic.cz/files/CORS/projects-bar/basic-datafile.json"
    ],
    "flavor": "basic"
  },
  
]

filtrovani treba podle isNg==false (tj. stare omnibary) se zobrazenim presmerovani:

cat omnibar.json | jq --slurp -r '.[] | select(.isNg == false) | .url + " -> " + .finalUrl'
https://stats.nic.cz -> https://stats.nic.cz/
https://dnssec-validator.cz -> https://www.dnssec-validator.cz/
https://haas.nic.cz -> https://haas.nic.cz/
https://stats.labs.nic.cz -> https://stats.nic.cz/
http://bezpecnedomeny.cz -> https://www.dnssec.cz/
https://dnssec.cz -> https://www.dnssec.cz/
https://napul.cz -> https://www.napul.cz/
http://unsigned.cz -> https://www.dnssec.cz/
https://nic.cz/konference-it/##Photo -> https://www.nic.cz/konference-it/
https://jak-funguje-dns.cz -> https://jak-funguje-dns.cz/
https://jakfungujedns.cz -> https://jakfungujedns.cz/
https://nic.cz/overeni -> https://www.nic.cz/overeni/
https://bird.nic.cz -> https://bird.network.cz/
https://bird.network.cz -> https://bird.network.cz/
https://blog.nic.cz -> https://blog.nic.cz/
https://en.blog.nic.cz -> https://en.blog.nic.cz/
https://honeymap.cz -> https://haas.nic.cz/stats/

geek flavor pouzivaji zrejme jen knoti:

$ cat omnibar.json | jq --slurp -r '.[] | select(.flavor == "geek") | .url'
https://knot-dns.cz
https://knot-resolver.cz

a snazil se BIRD, ale ta geek-lista.js (uz?) neexistuje a omnibar misto toho nacetl defaultni basic:

{
  "url": "https://bird.nic.cz",
  "finalUrl": "https://bird.network.cz/",
  "included": true,
  "isNg": false,
  "datafiles": [
    "https://www.nic.cz/files/CORS/projects-bar/basic-datafile.json",
    "https://www.nic.cz/files/CORS/projects-bar/geek-lista.js"
  ],
  "flavor": "basic"
}
{
  "url": "https://bird.network.cz",
  "finalUrl": "https://bird.network.cz/",
  "included": true,
  "isNg": false,
  "datafiles": [
    "https://www.nic.cz/files/CORS/projects-bar/basic-datafile.json",
    "https://www.nic.cz/files/CORS/projects-bar/geek-lista.js"
  ],
  "flavor": "basic"
}
Edited
omnibarcheck.js 1.72 KB
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment