List items
Items from the current list are shown below.
Gecko
19 Jan 2024 : Day 143 #
Over the last couple of days now I've been building versions of gecko and its related packages that support the LOAD_FLAGS_FROM_EXTERNAL flag. According to comments in the code, when this flag is set the engine considers the request to be the result of user-interaction, so I'm hoping this change will help ensure the >Sec-Fetch headers are given the correct values.
But of course not all page loads are triggered externally. There are other flags to identify other types of user interaction which I'll need to fix as well. But one thing at a time. First of all I need to find out if the changes I've made to the code are having any effect.
The changes affect sixteen packages and comprise 771 MiB of data in all (including the debug packages), which I'm now copying over to my device so I can install and test them. Just doing this can take quite a while!
[...]
I've installed them and run them and they don't fix the problem. But as soon as I ran them I remembered that I've not yet hooked up things on the QML side to make them work. I still have a bit more coding to do before we'll see any good results.
[...]
I've updated the QML so that the flag is now passed in for the routes I could find that relate to page loads being triggered externally. There are some potential gotchas here: am I catching all of the situations in which this can happen? Are some of the paths used by other routes as well, such as entering a URL at the address bar? Once I've confirmed that these changes are having an effect I'll need to go back and check these other things as well.
First things first. Let's find out whether the flag is being applied in the case where a URL is passed on the command line.
So it's disappointing to discover that the page is still not rendering. Why is could this be?
I set the user agent to the ESR 91 value I used previously:
The last thing I see before I head to bed is the DuckDuckGo logo staring back at me from the screen. There is a glitch with search though. The search action works, in the sense that it takes me to the result page, but no results are shown. So there's still some more work to be done.
Nevertheless I'll sleep a lot easier tonight after this progress today.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
But of course not all page loads are triggered externally. There are other flags to identify other types of user interaction which I'll need to fix as well. But one thing at a time. First of all I need to find out if the changes I've made to the code are having any effect.
The changes affect sixteen packages and comprise 771 MiB of data in all (including the debug packages), which I'm now copying over to my device so I can install and test them. Just doing this can take quite a while!
[...]
I've installed them and run them and they don't fix the problem. But as soon as I ran them I remembered that I've not yet hooked up things on the QML side to make them work. I still have a bit more coding to do before we'll see any good results.
[...]
I've updated the QML so that the flag is now passed in for the routes I could find that relate to page loads being triggered externally. There are some potential gotchas here: am I catching all of the situations in which this can happen? Are some of the paths used by other routes as well, such as entering a URL at the address bar? Once I've confirmed that these changes are having an effect I'll need to go back and check these other things as well.
First things first. Let's find out whether the flag is being applied in the case where a URL is passed on the command line.
$ gdb sailfish-browser [...] b DeclarativeTabModel::newTab Breakpoint 1 at 0x6faf8: DeclarativeTabModel::newTab. (2 locations) (gdb) r https://www.duckduckgo.com [...] Thread 1 "sailfish-browse" hit Breakpoint 1, DeclarativeTabModel::newTab (this=0x55559d7b80, url=..., fromExternal=true) at ../history/declarativetabmodel.cpp:196 196 return newTab(url, 0, 0, false, fromExternal); (gdb) p fromExternal $1 = true (gdb) b EmbedLiteViewChild::RecvLoadURL Breakpoint 2 at 0x7fbcb105f0: file mobile/sailfishos/embedshared/ EmbedLiteViewChild.cpp, line 482. (gdb) n Thread 8 "GeckoWorkerThre" hit Breakpoint 2, mozilla::embedlite:: EmbedLiteViewChild::RecvLoadURL (this=0x7f88690d50, url=..., aFromExternal=@0x7f9f3d3598: true) at mobile/sailfishos/embedshared/EmbedLiteViewChild.cpp:482 482 { (gdb) n 483 LOGT("url:%s", NS_ConvertUTF16toUTF8(url).get()); (gdb) n 867 ${PROJECT}/gecko-dev/obj-build-mer-qt-xr/dist/include/nsCOMPtr.h: No such file or directory. (gdb) n 487 if (Preferences::GetBool("keyword.enabled", true)) { (gdb) n 493 if (aFromExternal) { (gdb) n 497 LoadURIOptions loadURIOptions; (gdb) p aFromExternal $2 = (const bool &) @0x7f9f3d3598: true (gdb) n 498 loadURIOptions.mTriggeringPrincipal = nsContentUtils::GetSystemPrincipal(); (gdb) n 499 loadURIOptions.mLoadFlags = flags; (gdb) p /x flags $4 = 0x341000 (gdb) p /x nsIWebNavigation::LOAD_FLAGS_FROM_EXTERNAL $5 = 0x1000 (gdb) p /x (flags & nsIWebNavigation::LOAD_FLAGS_FROM_EXTERNAL) $6 = 0x1000Great! The flag is being set in the user-interface and is successfully passing through from sailfish-browser to qtmozembed and on to the EmbedLite wrapper for gecko itself. Let's see if it makes it all the way to the request header methods.
(gdb) b SecFetch::AddSecFetchUser Breakpoint 3 at 0x7fbba7b680: file dom/security/SecFetch.cpp, line 333. (gdb) b GetLoadTriggeredFromExternal Breakpoint 4 at 0x7fb9d3430c: GetLoadTriggeredFromExternal. (5 locations) (gdb) c Continuing. [New LWP 8921] Thread 8 "GeckoWorkerThre" hit Breakpoint 4, nsILoadInfo:: GetLoadTriggeredFromExternal (this=0x7f88eabce0) at ${PROJECT}/obj-build-mer-qt-xr/dist/include/nsILoadInfo.h:664 664 ${PROJECT}/obj-build-mer-qt-xr/dist/include/nsILoadInfo.h: No such file or directory. (gdb) n Thread 8 "GeckoWorkerThre" hit Breakpoint 4, mozilla::net::LoadInfo:: GetLoadTriggeredFromExternal (this=0x7f88eabce0, aLoadTriggeredFromExternal=0x7f9f3d3150) at netwerk/base/LoadInfo.cpp:1478 1478 *aLoadTriggeredFromExternal = mLoadTriggeredFromExternal; (gdb) p mLoadTriggeredFromExternal $9 = true (gdb) c [...] mozilla::dom::SecFetch::AddSecFetchUser (aHTTPChannel=aHTTPChannel@entry=0x7f88eabed0) at ${PROJECT}/obj-build-mer-qt-xr/dist/include/mozilla/DebugOnly.h:97 97 ${PROJECT}/obj-build-mer-qt-xr/dist/include/mozilla/DebugOnly.h: No such file or directory. (gdb) 350 nsAutoCString user("?1"); (gdb)The Sec-Fetch-User flag is now being set correctly. Hooray! It's clear from this that the LOAD_FLAGS_FROM_EXTERNAL flag is making it through all the way to the place where the flag values are set. So let's check what the actual values are using the debug output on the console:
[ Request details ------------------------------------------- ] Request: GET status: 200 OK URL: https://duckduckgo.com/ [ Request headers --------------------------------------- ] Host : duckduckgo.com User-Agent : Mozilla/5.0 (X11; Linux aarch64; rv:91.0) Gecko/20100101 Firefox/91.0 Accept : text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Language : en-GB,en;q=0.5 Accept-Encoding : gzip, deflate, br Connection : keep-alive Upgrade-Insecure-Requests : 1 Sec-Fetch-Dest : document Sec-Fetch-Mode : navigate Sec-Fetch-Site : none Sec-Fetch-User : ?1 If-None-Match : "65a81e81-4858"That looks like just the set of Sec-Fetch flags we wanted.
So it's disappointing to discover that the page is still not rendering. Why is could this be?
I set the user agent to the ESR 91 value I used previously:
"duckduckgo.com": "Mozilla/5.0 (Mobile; rv:91.0) Gecko/91.0 Firefox/91.0"Close the browser; clear out the cache; try again:
$ rm -rf ~/.local/share/org.sailfishos/browser/.mozilla/cache2/ \ ~/.local/share/org.sailfishos/browser/.mozilla/startupCache/ \ ~/.local/share/org.sailfishos/browser/.mozilla/cookies.sqlite $ sailfish-browser https://duckduckgo.com/
The last thing I see before I head to bed is the DuckDuckGo logo staring back at me from the screen. There is a glitch with search though. The search action works, in the sense that it takes me to the result page, but no results are shown. So there's still some more work to be done.
Nevertheless I'll sleep a lot easier tonight after this progress today.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
Comments
Uncover Disqus comments