List items
Items from the current list are shown below.
Gecko
10 Jan 2024 : Day 134 #
I was getting a little despondent trying to fix DuckDuckGo on ESR 91, but the tests I performed yesterday have invigorated me somewhat. It's now clear that there's a decent quantity of accesses being made when using ESR 91 and that wouldn't be happening unless at least some of the data was being interpreted as HTML.
But it's also clear that not everything is as it should be: on ESR 78 there are a mixture of SVG and PNG files being downloaded to provide the images on the page. In contrast, on ESR 91, there are no images being downloaded at all. There are two possible reasons for this I can think of:
First, there are some extraneous accesses in the list that have nothing to do with DuckDuckGo but are a consequence of gecko collecting settings data after the profile was wiped; lines like this:
Second, I noticed that some files appear to be the same but located at different URLs. For example, this can be found in the ERS 78 list:
Third, it then sorts the results alphabetically so that any identical lines in both files will appear in the same order. If there are any matching lines, this will make any diff of the two much cleaner.
Finally the command then performs a side-by-side diff on the result. Here's all of that put together:
Let's concentrate on the mobile version of the page first. Here's the diff:
For completeness let's do the same for the desktop collections:
There might be something in this, but initially I'm more interested in the mobile version of the site and there the overlap is far less. However, now that that I have the URLs, one thing I can do is download all of the files and try to use them to recreate the site on my own server. It's possible this will give better results than saving out the files from the desktop browser, so I'll be giving that a go tomorrow.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
But it's also clear that not everything is as it should be: on ESR 78 there are a mixture of SVG and PNG files being downloaded to provide the images on the page. In contrast, on ESR 91, there are no images being downloaded at all. There are two possible reasons for this I can think of:
- DuckDuckGo is serving pages that don't contain any images. Seems unlikely, but nevertheless a possibility.
- There are images but ESR 91 isn't turning them in to access requests. I have no idea why this might happen, yet it still feels the more likely of the two scenarios.
First, there are some extraneous accesses in the list that have nothing to do with DuckDuckGo but are a consequence of gecko collecting settings data after the profile was wiped; lines like this:
https://firefox.settings.services.mozilla.com/v1/buckets/monitor/collections/ changes/records?collection=hijack-blocklists&bucket=mainSo the command filters all of the lines that don't include duckduckgo.
Second, I noticed that some files appear to be the same but located at different URLs. For example, this can be found in the ERS 78 list:
https://duckduckgo.com/font/ProximaNova-ExtraBold-webfont.woff2While this can be found in the ESR 91 list:
https://duckduckgo.com/static-assets/font/ProximaNova-Reg-webfont.woff2These are both font files; they must surely be the same file, right? But they're located in different folders. So the command then strips the URLs down to the leafnames.
Third, it then sorts the results alphabetically so that any identical lines in both files will appear in the same order. If there are any matching lines, this will make any diff of the two much cleaner.
Finally the command then performs a side-by-side diff on the result. Here's all of that put together:
$ diff --side-by-side \ <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "duckduckgo" log1.txt) | sort) \ <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "duckduckgo" log2.txt) | sort)When I came up with this approach I thought it would give amazing results, but in practice it's not as exciting as I was hoping for.
Let's concentrate on the mobile version of the page first. Here's the diff:
$ diff --side-by-side \ <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "duckduckgo" ddg-urls-esr78-mobile.txt) | sort) \ <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "duckduckgo" ddg-urls-esr91-mobile.txt) | sort) 1-monster-v2--no-animation.svg | 18040-1287342b1f839f70.js 2-ghost-v2.svg | 38407-070351ade350c8e4.js 3-bathtub-v2--no-animation.svg | 39337-cd8caeeff0afb1c4.js 4-alpinist-v2.svg | 41966-c9d76895b4f9358f.js arrow.svg | 55015-29fec414530c2cf6.js b.9e45618547aaad15b744.js | 55672-19856920a309aea5.js d.01ff355796b8725c8dad.js | 61754-29df12bb83d71c7b.js duckduckgo85.js | 6a4833195509cc3d.css h.2d6522d4f29f5b108aed.js | 703c9a9a057785a9.css hi?7857271&b=firefox&ei=true&i=false&[...] | 81125-b74d1b6f4908497b.js l.656ceb337d61e6c36064.js | 93432-ebd443fe69061b19.js logo_homepage.alt.v109.svg | 94623-d5bfa67fc3bada59.js logo_homepage.normal.v109.svg | 95665-30dd494bea911abd.js o.2988a52fdfb14b7eff16.css | a2a29f84956f2aac.css p.f5b58579149e7488209f.js | _app-ce0b94ea69138577.js post3.html | _buildManifest.js > c89114cfe55133c4.css > ed8494aa71104fdc.css > f0b3f7da285c9dbd.css > framework-f8115f7fae64930e.js > home-34dda07336cb6ee1.js > main-17a05b704438cdd6.js > ProximaNova-Bold-webfont.woff2 ProximaNova-ExtraBold-webfont.woff2 ProximaNova-ExtraBold-webfont.woff2 > ProximaNova-RegIt-webfont.woff2 ProximaNova-Reg-webfont.woff2 ProximaNova-Reg-webfont.woff2 ProximaNova-Sbold-webfont.woff2 ProximaNova-Sbold-webfont.woff2 s.b49dcfb5899df4f917ee.css | _ssgManifest.js teaser-2@2x.png | webpack-7358ea7cdec0aecf.js ti.b07012e30f6971ff71d3.js < tl.3db2557c9f124f3ebf92.js < u.a3c3a6d4d7bf9244744d.js <Only three files are shared across the two collections. The remaining 22 and 27 files respectively are apparently different. I was honestly hoping for there to be more similarity.
For completeness let's do the same for the desktop collections:
$ diff --side-by-side \ <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "duckduckgo" ddg-urls-esr78-desktop.txt) | sort) \ <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "duckduckgo" ddg-urls-esr91-d.txt) | sort) 18040-1287342b1f839f70.js 18040-1287342b1f839f70.js 38407-070351ade350c8e4.js 38407-070351ade350c8e4.js 39337-cd8caeeff0afb1c4.js 39337-cd8caeeff0afb1c4.js 41966-c9d76895b4f9358f.js 41966-c9d76895b4f9358f.js 48292.8c8d6cb394d25a15.js < 55015-29fec414530c2cf6.js 55015-29fec414530c2cf6.js 55672-19856920a309aea5.js 55672-19856920a309aea5.js 61754-29df12bb83d71c7b.js 61754-29df12bb83d71c7b.js 6a4833195509cc3d.css 6a4833195509cc3d.css 703c9a9a057785a9.css 703c9a9a057785a9.css 81125-b74d1b6f4908497b.js 81125-b74d1b6f4908497b.js 93432-ebd443fe69061b19.js 93432-ebd443fe69061b19.js 94623-d5bfa67fc3bada59.js 94623-d5bfa67fc3bada59.js 95665-30dd494bea911abd.js 95665-30dd494bea911abd.js a2a29f84956f2aac.css a2a29f84956f2aac.css add-firefox.f0890a6c.svg < _app-ce0b94ea69138577.js _app-ce0b94ea69138577.js app-protection-back-dark.png < app-protection-front-dark.png < app-protection-ios-dark.png < app-store.501fe17a.png < atb_home_impression?9836955&b=firefox[...] < _buildManifest.js _buildManifest.js burn@2x.be0bd36d.png < c89114cfe55133c4.css c89114cfe55133c4.css chrome-lg.a4859fb2.png < CNET-DARK.e3fd496e.png < dark-mode@2x.3e150d01.png < devices-dark.png < ed8494aa71104fdc.css ed8494aa71104fdc.css edge-lg.36af7682.png < email-protection-back-dark.png < email-protection-front-light.png < email-protection-ios-dark.png < f0b3f7da285c9dbd.css f0b3f7da285c9dbd.css firefox-lg.8efad702.png < flame.1241f020.png < flame@2x.40e1cfa0.png < flame-narrow.70589b7c.png < framework-f8115f7fae64930e.js framework-f8115f7fae64930e.js home-34dda07336cb6ee1.js home-34dda07336cb6ee1.js legacy-homepage-btf-dark.png < legacy-homepage-btf-mobile-dark.png < macos.61889438.png < main-17a05b704438cdd6.js main-17a05b704438cdd6.js night@2x.4ca79636.png < opera-lg.237c4418.png < page_home_commonImpression?2448534&[...] < play-store.e5d5ed36.png < ProximaNova-Bold-webfont.woff2 ProximaNova-Bold-webfont.woff2 ProximaNova-ExtraBold-webfont.woff2 ProximaNova-ExtraBold-webfont.woff2 ProximaNova-RegIt-webfont.woff2 ProximaNova-RegIt-webfont.woff2 ProximaNova-Reg-webfont.woff2 ProximaNova-Reg-webfont.woff2 ProximaNova-Sbold-webfont.woff2 ProximaNova-Sbold-webfont.woff2 safari-lg.8406694a.png < search-protection-back-light.png < search-protection-front-dark.png < search-protection-ios-dark.png < set-as-default.d95c3465.svg < _ssgManifest.js _ssgManifest.js UT-DARK-DEFAULT.6cd0020d.png < VERGE-DARK-DEFAULT.8850a2d2.png < webpack-7358ea7cdec0aecf.js webpack-7358ea7cdec0aecf.js web-protection-back-dark.png < web-protection-front-dark.png < web-protection-ios-dark.png < widget-big@2x.a260ccf6.png < widget-small@2x.07c865df.png < windows.477fa143.png < WIRED-DARK-DEFAULT.b4d48a49.png <Here we see something more interesting. There are 30 files the same across the two collections with 43 and 0 files being different respectively. In other words, the ESR 91 collection is a subset of the ESR 78 collection.
There might be something in this, but initially I'm more interested in the mobile version of the site and there the overlap is far less. However, now that that I have the URLs, one thing I can do is download all of the files and try to use them to recreate the site on my own server. It's possible this will give better results than saving out the files from the desktop browser, so I'll be giving that a go tomorrow.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
Comments
Uncover Disqus comments