You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.
opnsense / core Public
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Before you add a new report, we ask you kindly to acknowledge the following:
The Disable HTTP_REFERER enforcement check option in System > Settings > Enforcement does not make the web interface work when no referer header is sent. Using a browser that does not send the referer header causes a 400 Bad Request error.
I am new to OPNsense from a longtime user of pfSense. As such, my experience is with a new 21.1.4 installation yesterday. There are forum posts that indicate this problem began with the 21.1.1 release.
In the first blog post, the user indicates the problem manifests with Safari on iOS, but not with other browsers.
In my case, I am the developer of Privacy Browser, which does not send the referer header, even for same-domain requests. Because the referer header is security theater and has significant privacy implications, there is no way for a user to enable it in Privacy Browser. However, if I edit the code to send the referer header, then Privacy Browser can load the website just fine (as long as JavaScript is enabled, which is to be expected).
The text was updated successfully, but these errors were encountered:
sorenstoutner commented Apr 13, 2021My guess is that it is indeed some issue underneath, possibly with a system package inherited from upstream. I do not see any errors on the dashboard that would be helpful in troubleshooting.
You can test using Privacy Browser on Android if you like, or using Safari on iOS according to the forum post. If you do use Privacy Browser, you will need to enable JavaScript (see the guide inside the app for instructions). As I indicated in the original post, if I make one change in the code to enable the referer header, then it works fine.
sorenstoutner changed the title Disabling Referer Header Currently Broken Apr 13, 2021 sorenstoutner commented Apr 13, 2021If it is helpful for you, the 400 Bad Request error is also shown when the referer header is sent but JavaScript is disabled. This could indicate that something involved in the processing of the JavaScript added a dependency on the referer header with the 21.1.1 release.
kulikov-a commented Apr 13, 2021 •
hi!
can confirm (if related): when debugging is enabled on lighttpd and accessing GUI from safari new tab on iPhone it produces an entry in the log: lighttpd[24119] | (request.c.359) invalid header value -> 400
looks like upstream issue for me at first glance: lighttpd/lighttpd1.4@262561f
/CC the one and only @gstrauss :)
gstrauss commented Apr 14, 2021ISTR recently having a conversation somewhere about why a client sending an empty request header field-value (after removing leading and trailing whitespace) was poor behavior.
gstrauss commented Apr 14, 2021 •@kulikov-a found it, above. lighttpd/lighttpd1.4@262561f is on lighttpd master, but not yet part of an official release.
In discussion on IRC in #lighttpd, I pointed to https://redmine.stoutner.com/issues/37 where one poster wrote
Using a null value is probably not the best approach. Keys without values are considered malformed headers by some web servers leading to HTTP 400 errors.
The lighttpd patch in lighttpd/lighttpd1.4@262561f will be part of the next lighttpd release, lighttpd 1.4.60, likely released some time in the next few months, and ignores empty (non-HTTP/2-pseudo) request headers instead of rejecting them.
gstrauss commented Apr 14, 2021@sorenstoutner, while you may argue that ABNF in RFC 7230 for field values does allow an empty field-value, I can separately argue that servers may treat strange behavior as an attack and may choose to reject such requests.
The requests from Privacy Browser would be less suspicious if the Referer header were either omitted, or non-empty. However, if you choose to maintain such behavior in Privacy Browser, that might be used to fingerprint and reject the usage of Privacy Browser. Something to consider WRT your posts in https://redmine.stoutner.com/issues/37
sorenstoutner commented Apr 14, 2021 •Just to be clear, Privacy Browser does not send an empty Referer header. Privacy Browser sends no Referer header. You can see this at https://www.stoutner.com/the-referer-header/.
Due to limitations in Android's System WebView, Privacy Browser currently sends an empty X-Requested-With header, which is what is being discussed in https://redmine.stoutner.com/issues/37. This header will be entirely removed in the 4.x series of Privacy Browser with the release of Privacy WebView.
When I tested sending the Referer header the change also populated the X-Requested-With header again, so that was the root of the problem. It is my fault for not separating the two during testing. (Also, I had forgotten that some websites don't like empty headers, despite previously discussing it in a bug report, as this is the first time I have experienced the issue in my own browsing.) I do not know what the root of the problem is with Safari. Perhaps they are sending an empty header as well, or perhaps it is something else.
A web server can do whatever they like. Trust me when I say that Privacy Browser already does a lot of things much more invasive than sending an empty header, and that is only going to increase in the future. For example, wait until the default becomes to send no user agent. ;) However, I would consider it silly for a web server to think itself "under attack" because there is a header with no value. There are a lot of real attacks in the world that web servers need to protect themselves against, and this isn't one of them..
