List items
Items from the current list are shown below.
Blog
5 Jan 2024 : Day 129 #
Yesterday we were looking at the User Agent strings, which I updated for ESR 91, but which we concluded wasn't the cause of the poorly rendering websites I've been experiencing.
One site that I'm particularly disappointed at being broken — and which I also mentioned yesterday — is DuckDuckGo. As this is currently my preferred search engine (although I admit I'm on the look out for a new one) it would be rather convenient for me personally if it was working well with the Sailfish Browser.
There is a workaround, in that DuckDuckGo already provides a non-JavaScript version of their site at https://duckduckgo.com/html. This site already works just fine with ESR 91 and I admit it's these kinds of considerate touches that keep me going back to use the site. But that doesn't detract from the fact that the standard JavaScript enabled site really ought to be working just fine with ESR 91.
So today I want to investigate further what the problem is. I'm a bit short on time today, so not expecting to get to the bottom of it, but it would be good to make a start.
It's also worth recalling that there were some problems with getting DuckDuckGo to work on previous versions of the browser. Digging through the git logs I eventually hit this, in the embedlite-components repository, which is likely what I'm thinking of:
When rendering the site, the only output in the console log is the following:
Given the complete lack of rendering it's hard to even know whether the correct site is being transferred or not: it may be that the browser is getting stuck on a redirect or something mundane like that.
To check this we can set the EMBED_CONSOLE environment variable to the value "network". This is a special value that, as well as printing out full debug information, also outputs everything textual that's sent between the network and the browser. This can generate crazy quantities of debug output, so it's almost never a good idea to use it. But in this case where we're just checking the contents of a single page being downloaded it can be really helpful.
Let's give it a go.
In all of the output below I've made quite significant abridgements to prevent the output from filling up multiple browser pages. I've tried to remove only uninteresting parts (mostly, the returned HTML code).
As a separate test I also tried turning off JavaScript to see how this might affect the rendering, but in that case the site was clever enough to notice and redirected the browser to https://html.duckduckgo.com/, which of course then rendered just fine.
We can compare the output with JavaScript disabled to that generated from the plain HTML version of the page.
So I'm a bit stumped. At this point my head is spinning a bit; it's late here. But I'm frustrated and don't want to stop just yet. Maybe there's another — simpler — site that exhibits the same problem? If I could find something like that, it might help.
So to try to take this further I tested all of the pages from the Sailfish Browser test suite. I thought maybe something might show up as broken there that, if fixed, might also fix the DuckDuckGo page.
I was surprised to discover that most of these pages already work as expecting, including screen taps, SVGs and (some) videos. Audio wasn't working, neither was the file picker and perhaps most problematic the back button is also broken (although, weirdly, the page history seems to be working just fine).
So several things to fix, but none of the pages failed to render at all. So this also brought me no closer to figuring out what the problem might be with DuckDuckGo.
It's been a day of testing today it seems, rather than fixing. If I'm honest I'm a bit stumped and very frustrated right now. Maybe I need to sleep on it to try to think of another angle to approach it from.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
One site that I'm particularly disappointed at being broken — and which I also mentioned yesterday — is DuckDuckGo. As this is currently my preferred search engine (although I admit I'm on the look out for a new one) it would be rather convenient for me personally if it was working well with the Sailfish Browser.
There is a workaround, in that DuckDuckGo already provides a non-JavaScript version of their site at https://duckduckgo.com/html. This site already works just fine with ESR 91 and I admit it's these kinds of considerate touches that keep me going back to use the site. But that doesn't detract from the fact that the standard JavaScript enabled site really ought to be working just fine with ESR 91.
So today I want to investigate further what the problem is. I'm a bit short on time today, so not expecting to get to the bottom of it, but it would be good to make a start.
It's also worth recalling that there were some problems with getting DuckDuckGo to work on previous versions of the browser. Digging through the git logs I eventually hit this, in the embedlite-components repository, which is likely what I'm thinking of:
$ git log --grep "DuckDuckGo"
commit 04bf236f9a57bdd01d02d83f02e55b848a8ed1af (upstream/jb45659, origin/jb45659)
Author: David Llewellyn-Jones <david@flypig.co.uk>
Date: Fri Aug 14 16:24:22 2020 +0000
[embedlite-components] Ensure touch events also fire mouse clicks.
Fixes JB#45659
Gesture single taps were previously configured to send out mouse events
only if they were preceded by a touchstart event, and only if
preventDefault() wasn't applied to the event.
Other browsers send out the event independent of this. The difference
manifests itself if stopPropagation() is applied to the touch event,
which supresses the mouse events on our browser, but not on others.
For example, this meant that the input field of DuckDuckGo couldn't be
focussed, and also prevented the Google Maps touch controls from
working.
This change alters things so that the mouse event is sent out even if
single tap isn't preceeded by a touch event.
Please try to ignore the fact that back in 2020 I apparently didn't know how to spell either preceded or suppresses. I want to focus on the fact that today I'm experiencing something different on ESR 91: now the site just doesn't render at all.When rendering the site, the only output in the console log is the following:
[JavaScript Warning: "Cookie “_ga_NL3SRVXF92” will be soon rejected because it
has the “SameSite” attribute set to “None” or an invalid value, without the
“secure” attribute. To know more about the “SameSite“ attribute, read
https://developer.mozilla.org/docs/Web/HTTP/Headers/Set-Cookie/SameSite"
{file: "https://www.googletagmanager.com/gtag/js?id=G-NL3SRVXF92&l=dataLayer&cx=c"
line: 307}]
That's a warning not an error and even if it was an error it doesn't look like it would be too serious. Not serious enough to prevent the site from rendering at any rate. So this doesn't help us much.Given the complete lack of rendering it's hard to even know whether the correct site is being transferred or not: it may be that the browser is getting stuck on a redirect or something mundane like that.
To check this we can set the EMBED_CONSOLE environment variable to the value "network". This is a special value that, as well as printing out full debug information, also outputs everything textual that's sent between the network and the browser. This can generate crazy quantities of debug output, so it's almost never a good idea to use it. But in this case where we're just checking the contents of a single page being downloaded it can be really helpful.
Let's give it a go.
In all of the output below I've made quite significant abridgements to prevent the output from filling up multiple browser pages. I've tried to remove only uninteresting parts (mostly, the returned HTML code).
$ EMBED_CONSOLE="network" MOZ_LOG="EmbedLite:5" sailfish-browser \
https://duckduckgo.com/
[...]
[JavaScript Error: "Unexpected event profile-after-change" {file:
"resource://gre/modules/URLQueryStrippingListService.jsm" line: 228}]
observe@resource://gre/modules/URLQueryStrippingListService.jsm:228:12
[...]
CONSOLE message:
OpenGL compositor Initialized Succesfully.
Version: OpenGL ES 3.2 V@415.0 (GIT@248cd04, I42b5383e2c, 1569430435)
(Date:09/25/19)
Vendor: Qualcomm
Renderer: Adreno (TM) 610
FBO Texture Target: TEXTURE_2D
[Parent 11746: Unnamed thread 7bfc002670]: I/EmbedLite WARN: EmbedLite::virtual
void* mozilla::embedlite::EmbedLitePuppetWidget::GetNativeData(uint32_t):127
EmbedLitePuppetWidget::GetNativeData not implemented for this type
JSScript: ContextMenuHandler.js loaded
JSScript: SelectionPrototype.js loaded
JSScript: SelectionHandler.js loaded
JSScript: SelectAsyncHelper.js loaded
JSScript: FormAssistant.js loaded
JSScript: InputMethodHandler.js loaded
EmbedHelper init called
Available locales: en-US, fi, ru
Frame script: embedhelper.js loaded
[ Request details ------------------------------------------- ]
Request: GET status: 200 OK
URL: https://duckduckgo.com/
[ Request headers --------------------------------------- ]
Host : duckduckgo.com
User-Agent : Mozilla/5.0 (X11; Linux aarch64; rv:91.0) Gecko/20100101
Firefox/91.0
Accept : text/html,application/xhtml+xml,application/xml;q=0.9,
image/webp,*/*;q=0.8
Accept-Language : en-GB,en;q=0.5
Accept-Encoding : gzip, deflate, br
Connection : keep-alive
Upgrade-Insecure-Requests : 1
Sec-Fetch-Dest : document
Sec-Fetch-Mode : navigate
Sec-Fetch-Site : cross-site
If-None-Match : "65959a92-471e"
TE : trailers
[ Response headers -------------------------------------- ]
server : nginx
date : Wed, 03 Jan 2024 21:07:55 GMT
content-type : text/html; charset=UTF-8
content-length : 18206
vary : Accept-Encoding
etag : "65959a94-471e"
content-encoding : br
strict-transport-security : max-age=31536000
permissions-policy : interest-cohort=()
content-security-policy : default-src 'none' ; connect-src
https://duckduckgo.com https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; manifest-src https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; media-src https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; script-src blob: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com 'unsafe-inline' 'unsafe-eval' ;
font-src data: https://duckduckgo.com https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; img-src data: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; style-src https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com 'unsafe-inline' ; object-src 'none' ;
worker-src blob: ; child-src blob: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; frame-src blob: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; form-action https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; frame-ancestors 'self' ;
base-uri 'self' ; block-all-mixed-content ;
x-frame-options : SAMEORIGIN
x-xss-protection : 1;mode=block
x-content-type-options : nosniff
referrer-policy : origin
expect-ct : max-age=0
expires : Wed, 03 Jan 2024 21:07:54 GMT
cache-control : no-cache
X-Firefox-Spdy : h2
[ Document content -------------------------------------- ]
<!DOCTYPE html><html lang="en-US"><head><meta charSet="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1,
user-scalable=1 , viewport-fit=auto"/><link rel="preload"
href="/static-assets/font/ProximaNova-RegIt-webfont.woff2" as="font"
type="font/woff2" crossorigin="anonymous"/>
[...]
<script>window.onerror = function _onerror(msg, source, lineno, colno) {
var url = "https://improving.duckduckgo.com/t/" + "static_err_global?" +
Math.ceil(Math.random() * 1e7) + "&msg=" + encodeURIComponent(msg) +
"&url=" + encodeURIComponent(source) + "&pathname=" +
encodeURIComponent(window.location && window.location.pathname || "") +
"&line=" + lineno + "&col=" + colno;
try {
if (window.navigator.sendBeacon) {
window.navigator.sendBeacon(url);
} else {
var img = document.createElement("img");
img.src = url;
}
} catch (e) {// noop
}
};</script>
[...]
Document output truncated by 73144 bytes
[ Document content ends --------------------------------- ]
JavaScript error: chrome://embedlite/content/embedhelper.js, line 259:
TypeError: sessionHistory is null
CONSOLE message:
[JavaScript Error: "TypeError: sessionHistory is null"
{file: "chrome://embedlite/content/embedhelper.js" line: 259}]
receiveMessage@chrome://embedlite/content/embedhelper.js:259:29
There's clearly plenty of document data being downloaded. I'm not sure what to make of this. It doesn't look like the site content is being held back; there's no obvious redirect. Frankly it all looks in order.As a separate test I also tried turning off JavaScript to see how this might affect the rendering, but in that case the site was clever enough to notice and redirected the browser to https://html.duckduckgo.com/, which of course then rendered just fine.
We can compare the output with JavaScript disabled to that generated from the plain HTML version of the page.
$ EMBED_CONSOLE="network" MOZ_LOG="EmbedLite:5" sailfish-browser \
https://html.duckduckgo.com/
[...]
[W] unknown:0 - Unable to open bookmarks "/home/defaultuser/.local/share/
org.sailfishos/browser/bookmarks.json"
[...]
[Parent 14102: Unnamed thread 776c002670]: E/EmbedLite FUNC::static bool
GeckoLoader::InitEmbedding(const char*):230 InitEmbedding successfully
CONSOLE message:
[JavaScript Error: "Unexpected event profile-after-change" {file: "resource://gre/modules/URLQueryStrippingListService.jsm" line: 228}]
observe@resource://gre/modules/URLQueryStrippingListService.jsm:228:12
[...]
JSScript: ContextMenuHandler.js loaded
JSScript: SelectionPrototype.js loaded
JSScript: SelectionHandler.js loaded
JSScript: SelectAsyncHelper.js loaded
JSScript: FormAssistant.js loaded
JSScript: InputMethodHandler.js loaded
EmbedHelper init called
Available locales: en-US, fi, ru
Frame script: embedhelper.js loaded
[ Request details ------------------------------------------- ]
Request: GET status: 302 Found
URL: https://html.duckduckgo.com/
[ Request details ------------------------------------------- ]
Request: GET status: 200 OK
URL: https://html.duckduckgo.com/html/
[ Request headers --------------------------------------- ]
Host : html.duckduckgo.com
User-Agent : Mozilla/5.0 (X11; Linux aarch64; rv:91.0) Gecko/20100101
Firefox/91.0
Accept : text/html,application/xhtml+xml,application/xml;q=0.9,
image/webp,*/*;q=0.8
Accept-Language : en-GB,en;q=0.5
Accept-Encoding : gzip, deflate, br
Connection : keep-alive
Upgrade-Insecure-Requests : 1
Sec-Fetch-Dest : document
Sec-Fetch-Mode : navigate
Sec-Fetch-Site : cross-site
[ Response headers -------------------------------------- ]
server : nginx
date : Wed, 03 Jan 2024 21:31:43 GMT
content-type : text/html
content-length : 138
location : https://html.duckduckgo.com/html/
strict-transport-security : max-age=31536000
permissions-policy : interest-cohort=()
content-security-policy : default-src 'none' ; connect-src
https://duckduckgo.com https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; manifest-src https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; media-src https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; script-src blob: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com 'unsafe-inline' 'unsafe-eval' ;
font-src data: https://duckduckgo.com https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; img-src data: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; style-src https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com 'unsafe-inline' ; object-src 'none' ;
worker-src blob: ; child-src blob: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; frame-src blob: https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; form-action https://duckduckgo.com
https://*.duckduckgo.com
https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/
https://spreadprivacy.com ; frame-ancestors 'self' ;
base-uri 'self' ; block-all-mixed-content ;
x-frame-options : SAMEORIGIN
x-xss-protection : 1;mode=block
x-content-type-options : nosniff
referrer-policy : origin
expect-ct : max-age=0
expires : Thu, 02 Jan 2025 21:31:43 GMT
cache-control : max-age=31536000
x-duckduckgo-locale : en_GB
X-Firefox-Spdy : h2
[ Request headers --------------------------------------- ]
Host : html.duckduckgo.com
User-Agent : Mozilla/5.0 (X11; Linux aarch64; rv:91.0) Gecko/20100101
Firefox/91.0
Accept : text/html,application/xhtml+xml,application/xml;q=0.9,
image/webp,*/*;q=0.8
Accept-Language : en-GB,en;q=0.5
Accept-Encoding : gzip, deflate, br
Connection : keep-alive
Upgrade-Insecure-Requests : 1
Sec-Fetch-Dest : document
Sec-Fetch-Mode : navigate
Sec-Fetch-Site : cross-site
TE : trailers
[ Response headers -------------------------------------- ]
server : nginx
date : Wed, 03 Jan 2024 21:31:43 GMT
content-type : text/html; charset=UTF-8
vary : Accept-Encoding
server-timing : total;dur=14;desc="Backend Total"
strict-transport-security : max-age=31536000
permissions-policy : interest-cohort=()
content-security-policy : default-src 'none' ; connect-src
https://duckduckgo.com https://*.duckduckgo.com
[...]
x-frame-options : SAMEORIGIN
x-xss-protection : 1;mode=block
x-content-type-options : nosniff
referrer-policy : origin
expect-ct : max-age=0
expires : Wed, 03 Jan 2024 21:31:44 GMT
cache-control : max-age=1
x-duckduckgo-locale : en_GB
content-encoding : br
X-Firefox-Spdy : h2
[ Document content -------------------------------------- ]
Document output skipped, content-type non-text or unknown
[ Document content ends --------------------------------- ]
[ Document content -------------------------------------- ]
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!--[if IE 6]><html class="ie6" xmlns="http://www.w3.org/1999/xhtml"><![endif]-->
<!--[if IE 7]><html class="lt-ie8 lt-ie9" xmlns="http://www.w3.org/1999/xhtml"><![endif]-->
<!--[if IE 8]><html class="lt-ie9" xmlns="http://www.w3.org/1999/xhtml"><![endif]-->
<!--[if gt IE 8]><!--><html xmlns="http://www.w3.org/1999/xhtml"><!--<![endif]-->
<head>
<link rel="canonical" href="https://duckduckgo.com/" />
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0,
maximum-scale=3.0, user-scalable=1" />
<meta name="referrer" content="origin" />
<title>DuckDuckGo</title>
<link title="DuckDuckGo (HTML)" type="application/opensearchdescription+xml"
rel="search" href="//duckduckgo.com/opensearch_html_v2.xml" />
<link rel="icon" href="//duckduckgo.com/favicon.ico" type="image/x-icon" />
<link rel="apple-touch-icon" href="//duckduckgo.com/assets/logo_icon128.v101.png" />
<link rel="image_src" href="//duckduckgo.com/assets/logo_homepage.normal.v101.png" />
<link rel="stylesheet" media="handheld, all" href="//duckduckgo.com/dist/h.aeda52882d97098ab9ec.css" type="text/css"/>
</head>
<body class="body--home body--html">
[...]
</html>
[ Document content ends --------------------------------- ]
JavaScript error: chrome://embedlite/content/embedhelper.js, line 259:
TypeError: sessionHistory is null
CONSOLE message:
[JavaScript Error: "TypeError: sessionHistory is null"
{file: "chrome://embedlite/content/embedhelper.js" line: 259}]
receiveMessage@chrome://embedlite/content/embedhelper.js:259:29
JavaScript error: resource://gre/modules/LoginManagerChild.jsm, line 541:
NotFoundError: WindowGlobalChild.getActor: No such JSWindowActor 'LoginManager'
CONSOLE message:
[JavaScript Error: "NotFoundError: WindowGlobalChild.getActor: No such
JSWindowActor 'LoginManager'"
{file: "resource://gre/modules/LoginManagerChild.jsm" line: 541}]
forWindow@resource://gre/modules/LoginManagerChild.jsm:541:27
handleEvent@chrome://embedlite/content/embedhelper.js:433:29
The main difference in this second case is that now we have a 302 "Found" redirect returned from the site, which then takes us to the JavaScript-free HTML version. Other than this, comparing the two unfortunately I don't see any obvious signs of why one might be working but the other not.So I'm a bit stumped. At this point my head is spinning a bit; it's late here. But I'm frustrated and don't want to stop just yet. Maybe there's another — simpler — site that exhibits the same problem? If I could find something like that, it might help.
So to try to take this further I tested all of the pages from the Sailfish Browser test suite. I thought maybe something might show up as broken there that, if fixed, might also fix the DuckDuckGo page.
I was surprised to discover that most of these pages already work as expecting, including screen taps, SVGs and (some) videos. Audio wasn't working, neither was the file picker and perhaps most problematic the back button is also broken (although, weirdly, the page history seems to be working just fine).
So several things to fix, but none of the pages failed to render at all. So this also brought me no closer to figuring out what the problem might be with DuckDuckGo.
It's been a day of testing today it seems, rather than fixing. If I'm honest I'm a bit stumped and very frustrated right now. Maybe I need to sleep on it to try to think of another angle to approach it from.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
Comments
Uncover Disqus comments