Request for enhanced cookie management

[ghost]

I feel a bit uneasy about the cookie management in HTTPSB. Quite obviously, the script and plugin settings in HTTPSB override the respective settings in Chrome. But what about cookie settings?

From my point of view, 3 things are missing or unclear:

1. HTTPSB makes no explicit distinction between permanent and session cookies, IMHO.
2. It has no option to globally allow session cookies. Many sites require session cookies in order to work properly. Such an option should ideally be complemented with a switch to remove them after x minutes.
3. It's unclear to me how the cookie management in HTTPSB interacts with the cookie settings in Chrome and with other extensions like Vanilla. This also applies to DOM (or local) storage. What takes precedence?

[gorhill]

I am going to do some reading and testing today about session cookies.

I installed Vanilla and and looked into it, and like HTTPSB, it doesn't modify user settings.

But while HTTPSB actively prevents cookies from reaching their intended destination when not whitelisted, Vanilla list and optionally delete them from the machine, i.e, a cookie which is not whitelisted by Vanilla will still reach its intended destination (unless of course they are delete explicitly by user, or through a timer.)

[gorhill]

From Wikipedia:

"A user's session cookie ... for a website exists in temporary memory only while the user is reading and navigating the website. ... Web browsers normally delete session cookies when the user closes the browser."

I did a test here, and session cookies are not deleted by chromium. Example in my list of session cookies still reported after I closed chromium multiple times, including all opened web pages, and uninstalled all extensions (except HTTPSB):

ads.yahoo.com/{testbounce}
.meteomedia.com/{s_sess}
groups.google.com/{groupsloginpref}

So this part is important: "Web browsers normally delete session cookies".

I trust nothing out there, and the above findings vindicate this stance: session cookies in chromium are not all deleted as expected, which means that if I were to allow session cookies, it would be as good as allowing user tracking. So my current position on this is to not change the current behavior: no special treatment for session cookies.

[gorhill]

Just found this:

Session Cookies not cleared when Chrome processes closed

Above bug is marked as "won't fix", which at this point makes me lean toward "won't fix" for an "option to globally allow session cookies" in HTTPSB.

[ghost]

Raymond, thanks for looking into this!

I am going to do some reading and testing today about session cookies.

I installed Vanilla and and looked into it, and like HTTPSB, it doesn't
modify user settings.

But while HTTPSB actively prevents cookies from reaching their intended
destination when not whitelisted, Vanilla list and optionally delete
them from the machine, i.e, a cookie which is not whitelisted by Vanilla
will still reach its intended destination (unless of course they are
delete explicitly by user, or through a timer.)

Yes, I know how Vanilla works, and it has a built-in configurable timer.
The reason why I'm suggesting to add an option to globally allow session
cookies: It simply makes web surfing and working with HTTPSB more
convenient. And with a built-in timer, privacy concerns are minimized.

[gorhill]

Currently, there is nothing that is allowed by HTTPSB which is not reflected in the matrix for a web page, meaning users have an exact view of what web sites are allowed to do in a web page.

Creating a special whitelisting of session cookies (which are, for all practical purpose, totally not different from normal cookies given they are not removed by the browser), will cause a condition where something will be allowed on a web page without being reflected in the matrix. For me it is unthinkable, and defeat the primary goal of HTTPSB.

Since session cookies are not that different from normal cookies (either are a threat to privacy), than if you think allowing them make your browsing more comfortable, I will argue that you could just whitelist cookie from the matrix (just like image for out-of-the-box HTTPSB).

[ghost]

From Wikipedia http://en.wikipedia.org/wiki/HTTP_cookie#Session_cookie:

"A user's session cookie ... for a website exists in temporary memory
only while the user is reading and navigating the website. ... Web
browsers normally delete session cookies when the user closes the browser."

I did a test here, and session cookies are not deleted by chromium.
Example in my list of session cookies still reported after I closed
chromium multiple times, including all opened web pages, and uninstalled
all extensions (except HTTPSB):

|ads.yahoo.com/{testbounce}
.meteomedia.com/{s_sess}
groups.google.com/{groupsloginpref}
|

So this part is important: "Web browsers normally delete session cookies".

I understand. Although I've never tested if, e.g., Firefox deletes them
automatically.

I trust nothing out there, and the above findings vindicate this stance:
session cookies in chromium are not all deleted as expected, which means
that if I were to allow session cookies, it would be as good as allowing
user tracking. So my current position on this is to not change the
current behavior: no special treatment for for session cookies.

I think it's also a semantic problem. I've always configured Firefox and
Chrome in such a way, that I generally allow cookies only until the
browser is closed, and I do not allow third-party cookies at all. Thus,
all cookies (regardless if they are labelled as session cookies or not)
are session cookies for me as they are deleted anyhow when the browser
is closed (except a couple of permanent cookies allowed for specific
sites). Vanilla works similarly, it just has an additional timer and it
obviously respects my Chrome setting which forbids 3rd party cookies.

[gorhill]

@tlu1024 : "Thus, all cookies (regardless if they are labelled as session cookies or not) are session cookies for me as they are deleted anyhow when the browser is closed"

Than it's just a matter of whitelisting all cookies in HTTPSB if it is the way you prefer it, no need for a special option. HTTPSB completely disregard chromium settings re. cookies (HTTPSB deals only with chromium's javascript settings).

Both HTTPSB and Vanilla "respect" your Chrome settings for the simple reason that they both don't look at Chrome settings re. cookies.

[ghost]

@tlu1024 https://github.com/tlu1024 : "Thus, all cookies (regardless
if they are labelled as session cookies or not) are session cookies for
me as they are deleted anyhow when the browser is closed"

Than it's just a matter of whitelisting all cookies in HTTPSB if it is
the way you prefer it, no need for a special option. HTTPSB completely
disregard chromium settings re. cookies (HTTPSB deals only with
chromium's javascript settings).

Okay, that's a reasonable solution. I suggested that option -
considering that HTTPSB does provide some cookie control already - as it
would make, e.g., Vanilla superflous. Thus, HTTPSB would offer a
complete cookie control solution. But I'm fine with what you suggested.

[gorhill]

Just to summarize the issue for the record, let's write down where Vanilla and HTTPSB overlap and where they differ:

This is the current situation:

                                  HTTPSB        Vanilla    
delete unwanted cookies              yes            yes
delete unwanted session cookies      yes             no
block cookie traffic                 yes             no

This is what you wish:

                                  HTTPSB        Vanilla    
delete unwanted cookies              yes            yes
delete unwanted session cookies       no             no
block cookie traffic                 yes             no

Maybe it's the kind of issue that needs to be discussed among a wider audience, I am curious if there would be any arguments for or against that would make me reconsider or strengthen my current stance.

[ghost]

Just to summarize the issue for the record, let's write down where
Vanilla and HTTPSB overlap and where they differ:

Good idea!

This is the current situation:

| HTTPSB Vanilla
delete unwanted cookies yes yes
delete unwanted session cookies yes no

I think that's not correct. In my understanding, Vanilla deletes all
cookies which are not on the whitelist after x minutes as specified in
the timer.

block cookie traffic yes no
|

This is what you wish:

| HTTPSB Vanilla
delete unwanted cookies yes yes
delete unwanted session cookies no no
block cookie traffic yes no

No, that's not what I want! Perhaps my distinction between permanent and
session cookies was confusing. Let's rather call them cookies which are
only accepted until the browser closes or until they are deleted by a timer.

So that's what I wish: An option in HTTPSB which globally allows 1st-
party cookies but combined with a timer which deletes them after a
specified time. It should be possible to override this policy for
specific sites in the cookie column by either allowing them permanently
(equivalent with the whitelist in Vanilla) or by explicitly forbidding
them (equivalent with a blacklist not available in Vanilla) via the padlock.

[tharrisone]

I did not know that if the host isn't whitelisted the cookie is not accessible. That is excellent.

I have noticed that certain cookies that are not domains that are whitelisted are still in my cookie list after a long period of time. I think that the cookie check isn't working properly after I close/restart my browser. I went and looked at the hosts to make sure they weren't whitelisted, also I have yet to see any local storage deleted. I'm using google chrome v33(dev) in Linux.

Do you know of some sites that load you up with cookies and local storage so that I can test this?

EDIT: There's a few web storage test sites and after i removed the temp whitelist it did remove the local storage.

[gorhill]

@tharrisone : Yes, blocking outbound cookies unless the destination is whitelisted is the primary task of HTTPSB. A while ago I decided for convenience to slap on top of this task cookie removal from the browser, but I had to revise this, and in the end this results in rather complex code.

Here are various facts which I had to take into consideration:

• Page-permission means a hostname can be blacklisted in general, but whitelisted when visiting a particular web page, which means a cookie can end up being blacklisted and whitelisted at the same time. This forced me to put in more complex code to deal with cookie removal in this situation.
• Extensions may have to rely on cookies too (ex. LastPass), and since HTTP requests from extensions is not processed by HTTPSB (unless behind-the-scene is checked), HTTPSB can not just remove all cookies found which are not whitelisted, this has caused HTTPSB to violate the rule of not interfering with other extensions. To fix this I had to remove the "janitor" code which was collecting and removing all non-whitelisted cookies found on the machine.
• It also worries me that I would give a false sense of security to users: HTTPSB takes care of cookies and localStorage, but does not to take care of webSQL, indexedDB and plug-ins private storage (though blocking script and plugin prevent these existing stores from being remotely accessed). For these last two, there is no chromium API to remove them based on their origin. Chromium API for removal of local data is time-based, not origin-based, which means HTTPSB can't just wipe out everything without knowing whether it is being used.

Because of the above considerations, as it is now HTTPSB will remove only cookies and localStorage (if asked) which are found while visiting a site where cookies are blacklisted. Pre-exisiting cookies and extensions cookies are left alone.

Workaround is to let the user make an informed decision: to check "Keep local data only until I quit my browser" in chromium settings, which is actually even a better option than the clean-up HTTPSB was attempting to do (or the clean-up Vanilla tries), because it get rids of a lot more stuff which are a potential threat to privacy than just cookies (i.e. indexedDB, webSQL, Flash private storage).

[tharrisone]

Thanks, that explains a lot. I can at least still use Vanilla cookie manager with no conflicts and both get the job done.

[gorhill]

@tlu1024

I am wiling to try to implement this, if this feature can be explained clearly to the user. So if I understand correctly:

A checkbox for "Do not block session cookies (except explicitly blacklisted ones)"

If "Delete cookie" is selected:

• Blacklisted normal/session cookies: remove without a second thought.
• Whitelisted normal/session cookies: keep them around even if not actively used.
• Graylisted session cookies: If above checkbox is not checked, threat as normal cookies
• Graylisted session cookies: If above checkbox is checked, keep them around until x minutes -- delay removal to the next cycle if still in use.

If I get this right, how can this become clear to the user what all the mean?

[gorhill]

Something I had not foreseen: in onBeforeSendHeaders(), I don't readily available have the information on whether a cookie is a session or not. I would need to parse the cookies, etc which at this point I don't want to do for fear of too much overhead -- onBeforeSendHeaders() is executed for every single request on the page, and cookie parsing etc. is not necessarily a lightweight task.

So I settled for this for now, an new option in "Settings"

• Delete non-blocked session cookies [???] minutes after last time they have been used.

At least this will bring back an expected feature of session cookie, which is that they are supposed to be destroyed in the short term (as noted above, Chromium doesn't destroy them when the browser is closed.)

Also, I reintroduced the cookie janitor, I added a note about how extensions relying on cookies might be affected if the hostnames from where cookies originate are blocked.

[ghost]

Raymond, I've been out of town for some time and I will be for the rest of the year. Please apologize that I won't be able to comment your remarks above until January (considering the changes you've implemented so far). Merry Xmas!

EDIT: Nevertheless a short question: You wrote that you "don't readily available have the information on whether a cookie is a session or not" since you would need to parse the cookies. On the other hand, you implemented the new option "Delete non-blocked session cookies ...". Isn't that a bit contradictory? I suggest that you change that to "Delete non-blocked cookies ..." , i.e. no distinction between session and permanent cookies. That's actually what I originally wanted before I confused you ;-) And that change would make Vanilla superfluous.

[gorhill]

"Isn't that a bit contradictory"

No, I don't have the information when OnBeforeSendHeaders() is called: the cookie information at this point is only a string in the cookie header, in the form name = value, nothing else. So there is no way for me to see whether a cookie is session or not at OnBeforeSendHeaders(). And even then, that would force me to parse the cookie string, which means unacceptable extra overhead at a time-critical point.

What I have in mind is to add (at the same time I add the clear cache feature) a clear stored data feature the same way. This will take care of what you are asking here, plus also the clearing of other information which cannot be deleted using any other mean: webStorage, Flash storage, etc.

[ghost]

"Isn't that a bit contradictory"

No, I don't have the information when |OnBeforeSendHeaders()| is called:
the cookie information at this point is only a string in the |cookie|
header, in the form |name = value|, nothing else. So there is no way for
me to see whether a cookie is session or not at |OnBeforeSendHeaders()|.
And even then, that would force me to parse the cookie string, which
means a unacceptable extra overhead at a time-critical point.

I understand. My point is that I don't see the need to distinguish
between session and permanent cookies here. I admit again that my first
comment was confusing/misleading. I wanted an option that ALL cookies
are treated as session cookies by HTTPSB (regardless how they are
labelled) and, thus, deleted after x minutes while it should be possible
to explicitly (permanently) allow or deny cookies for specific
domains/sites via the padlock.

So the optimal solution for me would be: Change that option in settings
to sometghing like "Allow cookies and delete them x minutes after last
time they have been used unless explicitly and permanently allowed in
the cookies column for specfic sites." (This should be combined with a
hint that 3rd party cookies should be blocked in the Chrome settings
unless you want to implement such a feature in HTTPSB itself.) If the
user choses that option the cookies column should automatically turn
pale green. (This should actually also apply to other types of storage -
see below.)

What I have in mind is to add (at the same time I add the clear cache
feature) a clear stored data feature the same way. This will take care
of what you are asking here, plus also the clearing of other information
which cannot be deleted using any other mean: webStorage, Flash storage,
etc.

There is a good overview of the types of storage here:
http://www.hotcleaner.com/web_storage.html

[gorhill]

I understand. My point is that I don't see the need to distinguish between session and permanent cookies here.

They are very different cookies. Normal cookies often exists to hold user preferences, and wiping these cookies will likely result in adverse effects, like having to select yet again your language/location/theme or whatever user prefs whenever one visit a particular favorite site. Session cookies are meant to be temporary, not to hold user prefs, so the worst adverse effect is to have to log in again.

I don't want angry users who did not fully understand the potential adverse effects of erasing normal cookies to ragequit the extension or for me to see non-stop reporting of expected adverse effects as bugs.

To complicate the matter further, I really wish I could provide for browser data what we now have for browser cache, but the effects of such a feature are potentially even more adverse than just removing cookies. Imagine: writing a post on a forum => all browser data being deleted before you post => click send => fail => lose post. => ragequit the extension with angry comments about how it breaks things. For this one feature the only approach I see now is to have a button in the popup menu which purpose is to let the user choose to clear browser data whenever he/she choose (and that include all cookies). This way there is no surprise, the user did it himself whenever he wanted.

[ghost]

I understand. My point is that I don't see the need to distinguish
between session and permanent cookies here.

They are very different cookies. Normal cookies often exists to hold
user preferences, and wiping these cookies will likely result in adverse
effects, like having to select yet again your language/location/theme or
whatever user prefs whenever one visit a particular favorite site.
Session cookies are meant to be temporary, not to hold user prefs, so
the worst adverse effect is to have to log in again.

I understand that. However, I still don't see the need to distinguish
between them. If a user accepts normal cookies that hold user
preferences, e.g., for a forum, he/she can make them permanent with the
padlock. Hence, those cookies would not be automatically deleted by
HTTPSB. On the other hand, if I accept normal cookies for a site - why
shouldn't I accept session cookies for it, too? No need to distinguish,
IMHO. That's why Vanilla doesn't make such a distinction, either. And if
you configure Chrome in such a way that cookies and other website data
are deleted at the end of the browser session (i.e. all cookies are
effectively treated as session cookies), you are also forced to define
exceptions for specific sites where you want cookies to "survive". This
would correspond to making them permanent in HTTPSB with the padlock.

Imagine: writing a post on a forum => all browser data being deleted
before you post => click send => fail => lose post.

This would not happen if the user had made cookies permanent for that
forum - see remarks above. And if he/she had not: The setting in HTTPSB
says:

"Delete non-blocked session cookies x minutes after last time they have
been used."

My interpretation is that cookies are not deleted if they are still in
use, i.e. the corresponding website is still loaded. If that
interpretation is correct, why should above scenario happen?

[gorhill]

However, I still don't see the need to distinguish between them

The potential adverse effects of deleting non-session cookies are what distinguish them. User preferences being wiped out is no fun.

if I accept normal cookies for a site - why shouldn't I accept session cookies for it, too?

Because as per definition session cookies are expected to expire in the near term. In Chromium they don't. Thus expectations are trumped. Hence HTTPSB offers a way for this definition to be respected.

if you configure Chrome in such a way that cookies and other website data are deleted at the end of the browser session

Exactly, there is no surprise here because the user expected them to all be deleted, per his choice in the browser settings and his action of closing the browser. But having them all deleted after x minutes, there is an element of surprise. Nobody will keeps a mental countdown of when their cookies expire.

I know you don't mind the browser data being cleared at x interval, but I have to keep in mind the average user, and not everybody will be pleased with adverse effects which will inevitably surprise in a bad way someone, the more the extension is widely used. If the user expressly clear the browser data using an HTTPSB button for that effect, he can't blame the extension, he did click the button. This is what I propose. This at least offers the convenience of not having to go in Chromium settings etc. and this button would clear all browser data, not just cookies.

I think the main divergence of opinion here is that you want HTTPSB to interpret differently the meaning of the "cookie" column: to accept/send all cookies regardless of the matrix state, but rather have the column control which cookies is not flushed from the browser data after x minutes.

I disagree with that view. My view is that an "allowed" cookie will reach its destination, while a "blocked" one wont. Like other request types, the column primary purpose is there to control how cookie information travels to remote site.

[ghost]

I think the main divergence of opinion here is that you want HTTPSB to interpret differently the meaning of the "cookie" column: to accept/send all cookies regardless of the matrix state, but rather have the column control which cookies is not flushed from the browser data after x minutes.

No, not "regardless of the matrix state". My idea is that every user still has the choice. He can still block all cookies if he wants that (cookie column is red) and this should still be the default behavior. But there should be an option in the settings that says something like this:

"Allow cookies and delete them x minutes after last time they have been used unless explicitly and permanently allowed in the cookies column for specfic sites."

If the user choses that option, the cookie column should turn, well, yellow. However, the user can still permanently allow or forbid cookies for specific sites/domains with the padlock. That's my idea - it's only an option which would completely replace extensions like Vanilla.

[gorhill]

If the user choses that option, the cookie column should turn, well, yellow

Interesting idea. The yellow color would take care of not confusing users which arise when we overload the definition of green. My first gut feeling is that would require lots of change. But I will keep assessing the idea and the side effects etc.

Still, about that button, it's still required if I want HTTPSB to easily allow deleting all browser local data (webSQL, localStorage, indexedDB, cookies, plugin data, cookie channels, etc.)

[ghost]

Thanks for thinking about this idea.

Regarding that button: I'm not quite sure how HTTPSB handles local storage right now. The settings says: "Delete contents of local storage of blocked hostnames". But what are blocked hostnames? The ones in the blacklists or the ones for which cookies are blocked? If the latter is true, I wonder if such a button is really needed. Because either if you block cookies by default (red column) you would also block local data; or you chose my suggested option (yellow column) which means that local data is also deleted after x minutes.

Another possibility is to offer a button that blocks local storage regardless of the cookie policy - corresponding to dom.storage.enabled=false in about:config in Firefox (http://kb.mozillazine.org/Dom.storage.enabled).

BTW, you didn't answer my question above: "My interpretation is that cookies are not deleted if they are still in
use, i.e. the corresponding website is still loaded. If that interpretation is correct, why should above scenario happen?" That's really interesting to know!

[gorhill]

But what are blocked hostnames? The ones in the blacklists or the ones for which cookies are blocked?

Good point, it is more accurately blocked cookies, not blocked hostnames (blocked = dark or light red, blacklisted = dark red).

When I say local storage, it is literal, I refer to localStorage, which does not comprise indexedDB, webSQL, plugin storage and whatnot, for the simple reason that there is no API in Chromium to delete these storages based on origin. It works for localStorage because I can do this by injecting code in the DOM (i.e I use a trick). But I can't use that trick for other types of storage -- because very strangely their content are not enumerable (and re. plugin it's even more opaque). I say strangely because I can't figure why indexedDB and webSQL would be designed that way in the first place (unless the designers didn't want the user to find out the content?) Or maybe I don't know yet how to use them.

For other types of storage, only the indiscriminate sledgehammer chrome.browsingData.remove* works.

"My interpretation is that cookies are not deleted if they are still in use, i.e. the corresponding website is still loaded. If that interpretation is correct, why should above scenario happen?"

Because I was planning to use the indiscriminate sledgehammer above to get rid of the cookies and all the other types of storage. I find that addressing only the cookies without the other types of storage is to not live up to the expectations cast by the extension in the user's mind.

[ghost]

I'm stubborn ;-) Just some additional remarks.

Regarding distingushing between session and normal cookies: Firefox doesn't do that.

https://support.mozilla.org/en-US/kb/settings-privacy-browsing-history-do-not-track#w_use-custom-settings-for-history
https://support.mozilla.org/en-US/kb/enable-and-disable-cookies-website-preferences

If you chose "Keep until I close Firefox: ", all cookies are deleted regardless if they are session or permanent (unless they are allowed as exceptions). I've never had any problems with that policy.

You wrote earlier: "Normal cookies often exists to hold user preferences, and wiping these cookies will likely result in adverse effects, ...". Correct, but cookies are blocked in HTTPSB by default. Thus, if a site requires cookies to store user preferences, the user has to whitelist them for that site anyhow. The problem is that many sites require normal cookies (not session cookies) in order to work properly (because the website admin carelessly does not distinguish between normal and session cookies, or simply because that website wants to track/recognize you if you come back to it). I do not want to whitelist this kind of cookies - but exactly this case is not covered in HTTPSB. That's why I think that an option is necessary that allows cookies temporarily while it's still possible to permanently allow cookies for specific sites (which hold user preferences). Above suggestion (that option with the yellow cookie column) would permit that.

[ghost]

I wonder if you still think about implementing what I suggested. I still believe that such a feature would be highly useful.

Many sites require that cookies (partly as session cookies, partly as normal cookies) are temporarily allowed in order to work properly. This case is not supported. If cookies are blocked those sites would not work, if cookies are allowed those sites could track you. My suggestion as noted in above posts: Adding an option (in combination with a timer) like "Allow cookies and delete them x minutes after last time they have been used unless explicitly and permanently allowed in the matrix for specific sites." If the user choses that option, the cookie column should turn yellow. Privacy impacts are minimal (particularly since cookies would be blocked for the domains in the integrated hosts files anyhow), and the user could still whitelist the cookie cell for a specific site if permanent cookies are required to store login credentials (or explicitly blacklist those cells for other sites).

This would not only make extensions like Vanilla superfluous but also makes cookie management in Chrome/Chromium much easier: If a site requires cookies (which do not hold user credentials) in order to work properly, you would have them to allow in HTTPSB and configure Chromium in such a way that cookies are deleted when the browser closes in order to not make them permanent. On the other hand, if a site requires permanent cookies (which hold user credentials), you have them to allow in HTTPSB and define an exception in the Chromium cookie settings so they are not deleted when the browser closes. By implementing above suggestion we could allow cookies by default in the Chromium settings and leave the cookie management completely to HTTPSB.

BTW: The message for "settingsDeleteBlockedLocalStoragePrompt" should - as discussed above - not refer to blocked hostnames but more precisely to hostnames for which cookies are blocked if I understood you correctly.

[ghost]

I know I'm a pain in the neck ;-) Anyway, I would like to illustrate my point with the following picture. It assumes that we have to allow cookies in HTTPSB for a website in order to make it work properly. Then the situation depends on 1. the Chrome cookie settings and 2. if the website is content with temporary cookies or requires permanent cookies (in order to store user credentials):

You see in B4 and C5 the two problematic situations. My suggestion above (an option that allows cookies temporarily and deletes them after x minutes while coloring the cookies column yellow) would take care of them. We could then generally allow cookies in the Chrome settings. This means:

1. The problem in column B would be solved. The user could be sure that all cookies are deleted after x minutes. Cookies for all sites on the ubiquitous lists would still be blocked anyhow. User tracking would still be impossible. (And explicitly blacklisting the cookie cell for specific sites would still be possible.)
2. The problem in column C would be solved, too. If the user wants to allow permanent cookies for a specific website he/she could explicitly whitelist the cookie cell for that site - without the need to also define an exception in the Chrome settings.

The main advantage of this proposal is that the outcome is much more predictable for the user as he/she does not need to "synchronize" his/her decision in HTTPSB with the Chrome settings. The complete cookie (and local storage) management would be done in HTTPSB. This would be much more manageable for the user.

Again, I suggest this as an option. No user would be forced to use it that way.

I don't know how difficult it is to implement it. But it would be great to have it :-)

[gorhill]

Sorry, I will take the time to share my thoughts on this after I finished working on other stuff.

[gorhill]

If I try to resume in one sentence the feature you wish (not necessarily what you are specifically suggesting above):

• Allow weakly blocked cookies but delete them after x minutes.
• Where "weakly blocked" means cookies blocked through inheritance, except when the cell from which it inherits is an explicit block rule for one of the domains which owns the cookie, or an explicit block rule for the cell itself.

New concept: "weakly".

• Weakly: block status inherited from top row (type or all cells).
• Strongly: block status inherited from self or domain or (ancestor domain(s) if any).

Consequences of such rules for cookies would be:

• For blacklisted cookies, or blacklisted sites the cookies will still be blocked (and optionally always deleted): feature already exists.
• For whitelisted sites the cookies will still be allowed: just as it is now.
• For weakly blocked cookies, the cookies will be allowed in the outgoing headers, but locally deleted after x minutes: new.

It would be a completely separate logic from allowed session cookies, i.e. nothing changes for allowed session cookies. It would just affects cookies which are currently blocked because they inherit their block status from the cookie cell in the top row, or the all cell. The yellow color is a good idea as it keeps the user accurately informed of what is happening under the hood.

Issues to address:

• What about cookies which are constantly refreshed? Their "last used" timestamp will always be reset, and thus never deleted. Example: Google's PREF cookie is quite often refreshed, even as a 3rd-party asset given how Google's domains are omnipresent.
• How do I translate this yellow color in a color-blind compatible matrix?
• To properly implement this new concept require some core code re-factoring in order to return whether a block is weak or strong (but I do like this new concept, I see it as naturally fitting the matrix filtering engine.)
• To come up with a setting phrased in a way that the feature can be understood by the user. Probably requires the introduction and definition of "weak" vs. "strong" blocking.

[ghost]

Raymond, thanks for considering this!

Let me address some points (not necessarily in the same order as in your
post).

• To come up with a setting phrased in a way that the feature can be
  understood by the user. Probably requires the introduction and
  definition of "weak" vs. "strong" blocking.

I guess that the distinction between "weak" and "strong" blocking would
be hard to understand for most users. Is it necessary to use such a
wording? Wouldn't it be sufficient to add an option (as I suggested in
an earlier post) like "Allow cookies and delete them x minutes after
last time they have been used unless explicitly and permanently
allowed in the matrix for specific sites/domains."?

• How do I translate this yellow color into a color-blind compatible
  matrix?

My idea would be cells with a separate marking, e.g., with a gingerly
fluted or dotted surface.

• What about cookies which are constantly refreshed? Their "last used"
  timestamp will always be reset, and thus never deleted. Example:
  Google's |PREF| cookie is quite often refreshed, even as a 3rd-party
  asset given how Google's domains are omnipresent.

Hm, wouldn't this problem be mitigated by honoring "block 3rd party
cookies" in the Chrome settings?

[gorhill]

"Allow cookies and delete them x minutes after last time they have been used unless explicitly and permanently allowed in the matrix for specific sites/domains."

Well this doesn't explain that cookies for blacklisted web sites will still be blocked. Let's see:

"Allow weakly blocked cookies and delete them x minutes after last time they have been used."

The "weakly" concept allows to condense a lot of information.

And then the help button would expand to explain that "weakly" means their block status is inherited from far, i.e. through the cookie or all cell in the matrix.

I like the idea of introducing new concepts if it makes sense. The way the matrix inheritance work in HTTPSB is kind of new, so introducing concepts related to it should not be too resisted either if it can replace mouthful sentences.

[ghost]

I see your point ... I was trying to make it less ... well ... abstract as I'm just a very simple-minded guy (and perhaps 2 or 3 others of your users) ;-)

BTW: When we say cookies we're talking about cookies and local storage, don't we?

[gorhill]

No, I mean only cookies, as local storage is not something which needs to be "blocked" on a wire, just deleted locally. Currently local storages are deleted only once, when the web page loads.

[gorhill]

I was trying to make it less ... well ... abstract

In that case, to be accurate your sentence would need to be something like (emphasis is what I added):

"Allow non-blacklisted blocked cookies of non-blacklisted domains and delete them x minutes after last time they have been used unless explicitly and permanently allowed in the matrix for specific sites/domains."

I suspect users would have to read this a couple of time before (hopefully) understanding what exactly it does.

If we use "weakly" (or whatever else term is better, I am non-native english-speaking too), then it becomes just a matter of explaining what the term means, then the feature will become clear with just a short sentence.

[ghost]

In that case, to be accurate your sentence would need to be something like (emphasis is what I added):

"Allow non-blacklisted blocked cookies of non-blacklisted domains and delete them x minutes after last time they have been used unless explicitly and permanently allowed in the matrix for specific sites/domains."

That's becoming a run-on sentence ;-) Perhaps we could split it in 2
sentences:

"Allow non-blacklisted blocked cookies of non-blacklisted
domains and delete them x minutes after last time they have been used.
You can still explicitly allow permanent cookies in the matrix for
specific sites/domains."

If we use "weakly" (or whatever else is better, I am non-native english-speaking too), then it becomes just a matter of explaining what the term means, then the feature will become clear with just a short sentence.

Yes, I think so. Although I'm still afraid that many people have trouble
dealing with such a "technical" approach. It would be interesting to
know what other users are thinking about this ...

[ghost]

No, I mean only cookies, as local storage is not something which needs to be "blocked" on a wire, just deleted locally. Currently local storages are deleted only once, when the web page loads.

I'm afraid that I don't understand the technical difference between
both. From my understanding local storage is kind of a super-cookie
that allows for permanent and session-only storage of user data, too.

[gorhill]

I'm afraid that I don't understand the technical difference between both

Cookies can exist in two states: "vector" (in transit on a network) or "point" (saved in a local storage). Local storages are only "point" -- they don't transit and thus there is no need to block their movement.

I don't see HTML5 localStorage really related the current issue. They will still be completely treated as they are treated now if I go forward with the feature here -- no change there.

[ghost]

Cookies can exist in two states: "vector" (in transit on a network) or "point" (saved in a local storage). Local storages are only "point" -- they don't transit and thus there is no need to block their movement.
But cookies have to be written (or saved) at first and read (and
possibly modified/updated) at the next visit of that website. Isn't this
also true for local storage?

I don't see HTML5 localStorage really related the current issue. They will still be completely treated as they are treated now if I go forward with the feature here -- no change there.

This is a (complete?)
overview of the variants of web storage in Chrome. Where do you assign
HTML5 storage in this picture? And what types of local storage are being
taken care of by HTTPSB?

[gorhill]

But cookies have to be written (or saved) at first and read (and possibly modified/updated) at the next visit of that website. Isn't this also true for local storage?

Ok, if you think there is an issue with local storage, I think it would be better to address it in another new issue. The issue here is to temporarily allow generically blocked cookies in net requests (I changed my mind about "weakly blocked", I prefer "generically blocked", it's closer to the definition) in order to allow some web sites to work without whitelisting cookies (for example, nytimes.com)

I don't understand why you see local storage linked to this particular issue here, I find debating local storage here is detracting from the issue at hand. HotCleaner deals with the storage aspects of cookies, the issue here is the network aspect.

[ghost]

I don't understand why you see local storage linked to this particular issue here, I find debating local storage here is detracting from the issue at hand.

Sorry for that. I'm mentioning it here because cookies and local storage
are obviously paired in the Chrome settings: If you allow cookies (for a
site or globally), you also allow local storage if I'm not completely
mistaken. (BTW: It's the same in Firefox unless you set
dom.storage.enabled=false)

If it is nevertheless possible to treat them separately - all the better!

[gorhill]

There is a HTTP header cookie, but there is no HTTP header for localStorage or whatever else storage flavor exists out there nowadays.

It's the blocking of the HTTP header cookie which prevents some sites from working properly, i.e. on my side when I click http://www.nytimes.com/2014/06/22/us/atlanta-civil-rights-museum.html, I am presented with a login page if I block HTTP cookie headers, but presented with the proper article if I allow HTTP cookie headers. My understanding it's what we are trying to address here: allow cookie but ensure that their content is short-lived, just so some web site still work yet are not able to use cookies as a tracking device.

[ghost]

My understanding it's what we are trying to address here: allow cookie but ensure that their content is short-lived, just so some web site still work yet are not able to use cookies as a tracking device.

Agreed! However, my concern is: If we agree that we have to allow cookies in the Chrome settings as a precondition to implement a cookie management in HTTPSB as we've been discussing (as blocking cookies in Chrome - row 3 in my image above - would mean that there is nothing to allow in HTTPSB), this means that we also allow local storage globally as both functionalities are tied together in Chrome. In other words, if HTTPSB only takes care that cookies are short-lived, local storage (aka "super-cookies") would survive and could track us. I don't think that's what we want. (Unless you come up with a completely new solution where the Chrome settings are not involved at all.)

BTW, we had discussed local storage before. I think this might be relevant here. But perhaps I'm misunderstanding something.

[gorhill]

Ok I see what you mean. These are two different issues, and I got confused because I focused on one specific. The other needs to be its own issue. The solution to the other is wild, as there is no way to specifically delete all local data by origin, i.e. once you call the API to delete local data, its the local data of all web sites which will be removed, and the only filter option is to delete all local data after a specific point in time in the past, not before a specific point in time in the past. Also, there is absolutely no way to relate this to the matrix, it's completely unrelated.

So to mix up both issues is pointless as the "Delete generically blocked cookies x minutes after they have been used" cannot be applied to local data. My focus here is to solve the nytimes.com, the local data issue must be entered in a separate issue, and the only available solution for this one is like the one for clearing the cache: to clobber all local data of all web sites at regular interval, with the potential inconvenience that you may loose data while doing something on a web site if it kicks in suddenly.