Electron Intercept HTTP request, response on BrowserWindow


#1

Hi, I am wondering if it is possible to create a BrowserWindow with a webview where the user navigate though links on the page being rendered by the BrowserWindow, and as the user interacts with the page be able to get every HTTP request/response being made by chromium, with details as to request/response headers, body and etc?


#2

You can get the HTTP response by listening to “did-get-response-details” for the <webview> tag.

// <webview id="webview">
var webViewEl = document.getElementById("webview"); 
webViewEl.addEventListener("did-get-response-details", function(details) {
    console.log(details); 
}); 

The “details” object will have information in the below format.

However, I’m not sure if you could get HTTP requests from event listeners.


#3

Thanks for the reply, now I am wondering if with native plugins for chromium I would be able to intercept the request.


#4

Theoretically possible!but you have to read the source code of chromium, that’s a nightmare,I think.


#5

Could you not use WebRequest to achieve this? I haven’t tried it yet, but it looks promising:

https://electron.atom.io/docs/api/web-request/


#6

You can intercept and cancel the unwanted request

session.defaultSession.webRequest.onBeforeRequest(['*://*./*'], function(details, callback) {
        
        var test_url = details.url;
        var check_block_list =/\.(gr|hk||fm|eu|it|es|is|net|ke|me||tz|za|zm|uk|us|in|com|de|fr|zw|tv|sk|se|php|pk|pl)\/ads?[\-_./\?]|(stats?|rankings?|tracks?|trigg|webtrends?|webtrekk|statistiche|visibl|searchenginejournal|visit|webstat|survey|spring).*.(com|net|de|fr|co|it|se)|cloudflare|\/statistics\/|torrent|[\-_./]ga[\-_./]|[\-_./]counter[\-_./\?]|ad\.admitad\.|\/widgets?[\-_./]?ads?|\/videos?[\-_./]?ads?|\/valueclick|userad|track[\-_./]?ads?|\/top[\-_./]?ads?|\/sponsor[\-_./]?ads?|smartadserver|\/sidebar[\-_]?ads?|popunder|\/includes\/ads?|\/iframe[-_]?ads?|\/header[-_]?ads?|\/framead|\/get[-_]?ads?|\/files\/ad*|exoclick|displayad|\ajax\/ad|adzone|\/assets\/ad*|advertisement|\/adv\/*\.|ad-frame|\.com\/bads\/|follow-us|connect-|-social-|googleplus.|linkedin|footer-social.|social-media|gmail|commission|adserv\.|omniture|netflix|huffingtonpost|dlpageping|log204|geoip\.|baidu|reporting\.|paypal|maxmind|geo\.|api\.bit|hits|predict|cdn-cgi|record_|\.ve$|radar|\.pop|\.tinybar\.|\.ranking|.cash|\.banner\.|adzerk|gweb|alliance|adf\.ly|monitor|urchin_post|imrworldwide|gen204|twitter|naukri|hulu.com|baidu|seotools|roi-|revenue|tracking.js|\/tracking[\-_./]?|elitics|demandmedia|bizrate|click-|click\.|bidsystem|affiliates?\.|beacon|hit\.|googleadservices|metrix|googleanal|dailymotion|ga.js|survey|trekk|visit_|arcadebanners?|visitor\.|ielsen|cts\.|link_|ga-track|FacebookTracking|quantc|traffic|evenuescien|roitra|pixelt|pagetra|metrics|[-_/.]?stats?[.-_/]?|common_|accounts\.|contentad|iqadtile|boxad|audsci.js|ebtrekk|seotrack|clickalyzer|youtube|\/tracker\/|ekomi|clicky|[-_/.]?click?[.-_/]?|[-_/.]?tracking?[.-_/]?|[-_/.]?track?[.-_/]?|ghostery|hscrm|watchvideo|clicks4ads|mkt[0-9]|createsend|analytix|shoppingshadow|clicktracks|admeld|google-analytics|-analytic|googletagservices|googletagmanager|tracking\.|thirdparty|track\.|pflexads|smaato|medialytics|doubleclick|cloudfront|-static|-static-|static-|sponsored-banner|static_|_static_|_static|sponsored_link|sponsored_ad|googleadword|analytics\.|googletakes|adsbygoogle|analytics-|-analytic|analytic-|googlesyndication|google_adsense2|googleAdIndexTop|\/ads\/|google-ad-|google-ad?|google-adsense-|google-adsense.|google-adverts-|google-adwords|google-afc-|google-afc.|google\/ad\?|google\/adv\.|google160.|google728.|_adv|google_afc.|google_afc_|google_afs.|google_afs_widget|google_caf.js|google_lander2.js|google_radlinks_|googlead|googleafc.|googleafs.|googleafvadrenderer.|googlecontextualads.|googleheadad.|googleleader.|googleleads.|googlempu.|ads_|_ads_|_ads|easyads|easyads|easyadstrack|ebayads|[.\-_/\?](ads?|clicks?|tracks?|tracking|logs?)[.\-_/]?(banners?|mid|trends|pathmedia|tech|units?|vert*|fox|area|loc|nxs|format|call|script|final|systems?|show|tag\.?|collect*|slot|right|space|taily|vids?|supply|true|targeting|counts?|nectar|net|onion|parlor|2srv|searcher|fundi|nimation|context|stats?|vertising|class|infuse|includes?|spacers?|code|images?|vers|texts?|work*|tail|track|streams?|ability||world*|zone|position|vertisers?|servers?|view|partner|data)[.\-_/]?/gi
        var check_white_list =/status|premoa.*.jpg|rakuten|nitori-net|search\?tbs\=sbi\:|google.*\/search|ebay.*static.*g|\/shopping\/product|aclk?|translate.googleapis.com|encrypted-|product|www.googleadservices.com\/pagead\/aclk|target.com|.css/gi;
        var block_me = check_block_list.test(test_url);
        var release_me = check_white_list.test(test_url);

        if(release_me){
            callback({cancel: false})
        }else if(block_me){
            callback({cancel: true});

        }else{
            callback({cancel: false})
        }

});

If you are expecting Adblock like feature check this : Resolved : Adblock for electron app


#7

I managed to handle this, for a webview. I needed to get the source only for the main document, and for all documents so I never detach the debugger, but you can easily adapt it for your needs.
The only problem I got is for the first url, at application startup : I wasn’t abel to attach the debugger soon enough to get the ‘Network.requestWillBeSent’ event, and ‘Network.getResponseBody’ is not working after ‘Network.responseReceived’, so I ended up reloading the page.

See https://chromedevtools.github.io/devtools-protocol/tot/Network/ for all available Network debugger commands.

const webview = this.$refs.webview
let firstShotReloaded = false

const attachDebugger = () => {
  const debug = webview.getWebContents().debugger
  debug.attach('1.1')
  debug.on('message', (event, method, params) => {
    if (!firstShotReloaded && method === 'Network.responseReceived') {
      // XXX did not find any other way for first page load
      firstShotReloaded = true
      webview.reload()
    }
    if (method === 'Network.requestWillBeSent') {
      if (params.request.url === webview.getURL()) {
        debug.sendCommand('Network.getResponseBody', { requestId: params.requestId }, (err, data) => {
          if (err.code === undefined) {
            // XXX may check data.base64encoded boolean and decode ? Maybe not here...
            // if (data.base64encoded) ... Buffer.from(data.body, 'base64');
            this.$store.dispatch('updateStaticSource', data.body)
          }
        })
      }
    }
  })
  debug.sendCommand('Network.enable')
  webview.removeEventListener('did-start-loading', attachDebugger)
}
webview.addEventListener('did-start-loading', attachDebugger)

#8

The new WebRequest object on a BrowserWindow seems all but inaccessible when the BrowserWindow is instanced. Assigning onHeadersReceived for the main window doesn’t make anything happen (it never fires) Trying to assign a handler to the defaultSession of a child BrowserWindow upon instancing

win.session.defaultSession.webRequest.onHeadersReceived([],onHeadersReceived);

just calls the fatal exit handler for Electron.

Did anyone ever get this to work?